Lecture 3: Regular languages and finite state machines

A language, in a formal sense, is simply a set of strings.  Languages can be defined explicitly, as in L = { the , a , an }, which is a language containing three strings, or implicitly, as in L = all valid English sentences (to the extent that the latter is well defined).
Languages can be finite or infinite.  The first example is finite.  The second is (probably) infinite, for instance:

• Bob watched Alice eat the apple.
• Claire watched Bob watch Alice eat the apple.
• Doug watched Claire watch Bob watch Alice eat the apple.
• Elif watched Doug watch Clair watch Bob watch Alice eat the apple.
• ... and so on

In this case, we've obtained an infinite language through recursion, and most people agree that recursion is a hallmark of human language (though I saw an article recently saying they found some obscure language that didn't have recursion).

A simpler way to get an infinite language is as follows:

• Bob is happy.
• Bob is very happy.
• Bob is very very happy.
• Bob is very very very happy.
• ... and so on

If we're being formal, we need some way of formalizing these notions of "and so on."  Regular expressions are a method of doing this.  They contain the following basic operations:

• /a/ means the string "a"
• /ab/ means match "a" followed by "b"
• /a*/ means a sequence of zero or more "a"s
• /a?/ means an optional "a"
• /a|b/ means "a" or "b"
• /^a/ means match "a" but only at the beginning of a string
• /a$/ means match "a" only at the end
• /a(b|c)/ means match "a" followed by one of "b" or "c"

These are the most basic operations.  In addition, some convenient short-hand is usually introduced:

• /a+/ means /aa*/ (i.e., one or more "a"s) -- note that "grep" doesn't support this: you have to use "egrep"
• /./ means any single character
• /[abcde]/ means /a|b|c|d|e/
• /[a-z]/ means /a|b|c|...|y|z/
• If you need to say "." or "*" or "+" put a "\" before it.

So how can we describe our "very very" language? How about /Bob is( very)* happy\./.  This is the infinite language that matches our example above.  (Note carefully where the spaces are.)  Exercise: write a regular expression for the "watch" language at the top!

One way of defining a "regular language" is "any language that can be described by a regular expression."
Note: regular languages are closed under: concatenation, intersection, disjunction, Kleene closure.



Another way to approach languages is through "machines" or "automata."  A finite state automata (FSA) is defined by:

1. A (finite) set of states
2. A set of labeled directed edges ("transitions") connecting states.  The labels are from the alphabet of our language.
3. A set of initial states (usually we assume just one)
4. A set of final states (usually we assume just one)

Note that cycles and self-loops are allowed!

You can think of an FSA as a machine that jumps around states, and emits the "letters" on the edges as it goes.  It can only stop in a final state.  This is the generative view.

The alternative view is that you give an FSA a string, and it tries to follow the corresponding edges, until is reaches a final state.  If it can reach a final state, it "accepts" the string.  If it cannot, it "rejects" it.  (Note that if one path leads to acceptance and another to rejection, then it accepts.)

What we defined above is a non-deterministic FSA.  A deterministic FSA is one in which two outgoing edges from a single state never have the same label.

An additional augmentation is to allow epsilon-transitions, namely, transitions that don't emit any words.

It's not too hard to verify that the set of languages that can be defined by an FSA is exactly the set of regular languages.  Moreover, adding epsilon transitions and non-determinism don't give you any extra power.

FSAs are very powerful despite being so simple.  They are useful (at least) for the following tasks:

• Grammars of person names
• Times, dates, emails, addresses, etc.
• Morphological analysis / segmentation

However, there is much linguistic structure they cannot capture, eg.:

• Center embedding: The mouse the cat chased died.
• Recursion of arbitrary depth: The man who [some sentence] is nice.

Here's an example for morphology:

• Cats -> cat +N +PL
• Cat -> cat +N +SG
• Cities -> city +N +PL
• Merging -> merge + V +present-participle
• Caught -> catch +V +past-participle

Why?  Spelling correction... translation (querer -- "want" --  can be realized as quiero or quieres)... hyphenation... pronunciation (hothouse)... etc...  Some languages do in morphology what English does in syntax:

• uyuyorum    I am sleeping
• uyuyorsun    you are sleeping
• uyuyor     he/she/it is sleeping
• uyuyoruz     we are sleeping
• uyuyorsunuz     you are sleeping
• uyuyorlar     they are sleeping
• uyuduk     we slept
• uyudukça     as long as (somebody) sleeps
• uyumalıyız     we must sleep
• uyumadan     without sleeping
• uyuman     your sleeping
• uyurken     while (somebody) is sleeping
• uyuyunca     when (somebody) sleeps
• uyutmak     to cause somebody to sleep
• uyutturmak     to cause (somebody) to cause (another) to sleep
• uyutturtturmak     to cause (somebody) to cause (some other) to cause (yet another) to sleep
• ... and so on

In this case, we don't want to just recognize, we want to transduce (aka "map").  So replace our transition symbols with input:output pairs.  We can then think of this machine as doing any of the following:

• Recognizing pairs of strings
• Mapping input string to (sets of) output strings
• Mapping output strings to (sets of) input strings

See the book for some examples of how to do simple English morphology with such an approach.

Don't discount this approach for being too simple, either...  people have done the following with finite state transducers:

• Transliteration
• Summarization
• Translation
• Question answering
• Dialogue agents
• Parsing (needs some extensions)
• ... pretty much everything :)