Lecture 12: First Order Logic

So far we've talked a lot about syntax. Of course, syntax isn't the end-all be-all of language processing: we want to "understand" language. Often this means mapping natural language sentences to some form of formal language sentences. This means we have to decide on a formal language. A common choice is first order logic (FOL).

Expressions in FOL take one of the following forms:

• Atomic symbols like "Hal"
• Relation symbols like "Eats(.,.)", for instance in "Eats(Hal, Sandwich)".  Relations can have arbitrary arity (i.e., take arbitrarily many arguments)
• Variables like "a" (see later for quantification)
• Combinations via and, or, not, implies, iff, etc.  Eg., Eats(Hal, Sandwich) implies not OnThePlate(Sandwich)
• Existential and universal quantifiers.  For example, "(exists a) Eats(Hal, a)" (which roughly means "Hal eats something.")
• Equality symbol.  For example, "(exists a) Eats(Hal, a) and (a = Sandwich)"

Of course a lot of this is redundant: you only really need negation and "and" and one quantifier and relations and equality and can derive the rest.

FOL is a nice language to map into because we know a lot about how to reason in FOL.  Reasoning is, of course, hard (take an AI class), but sort of do-able.

The general way in which FOL systems work is:

• We have a bunch of background knowledge expressed as "axioms" -- basically a database of FOL sentences
• We get a text, from which we derive new FOL sentences
• Someone issues a query, which we will try to "prove" given our collection of sentences.  A proof of a query is an answer to the question
• Someone might query "Who shot Lincoln?"
• We map this to something like "(exists a) Shot(a, Lincoln)"
• We try to prove this statement given our database, which might include some statement like "Shot(Booth, Lincoln)".  Thus, "a=Booth" is a "proof" of our query, and gives us the answer

The big problem is often interpretation: how to get from natural language to formal language.  Syntax is a bridge that will help us get there, but it's not everything.

The common assumption (which is not always true) is that language is compositional.  Namely, the meaning of a sentence is derived from the meanings of its parts.  For examples:

• Hal ate a sandwich with mayo.
• Parses to (Hal (ate (a sandwich (with mayo))))
• Bottom up, we can look at (mayo) and derive "(exists a) Mayo(a)"
• Then (sandwich (with mayo)) and derive "(exists b) Sandwich(b) and Contains(Sandwich, ___)" where ___ comes from the interpretation of (with mayo), yielding "(exists a,b) Mayo(a) and Sandwich(b) and Contains(a,b)"
• Then (a sandwich ...), takes what we had before and makes "b" unique, usually written as "(exists a, exists!b) ..."
• Then we take Hal and get "Hal"
• And finally, "x ate y" will give us something like "(exists e) Eat(e, x, y) and InThePast(e)" which when we plug in with x and y gives the final interpretation:
    (exists a, exists !b, exists e) Mayo(a) and Sandwich(b) and Contains(a,b) and Eat(e, Hal, b) and InThePast(e)

The hope is that this interpretation is agnostic to annoyances of syntax.  For example, if we instead wrote "The sandwich containing mayo was eaten by Hal", we would end up with the same interpretation.

In order to make this happen, we usually need a semantic grammar that tells us how to get from syntax to semantics.  The grammar will consist of rules that are akin to the type of rules we applied to do the semantic analysis of the above example sentence.

Key challenges with FOL:

• What are the right sets of symbols and relations?
• Where does the necessary background knowledge come from?
• You can try to get humans to do it for you: see the Cyc project
• Or you can try to extract it automatically: see ReadTheWeb or ODIE or TextRunner, for example
• How to get the semantic grammar
• You can hand code it (see verbnet and the unified verb index for a verb-centric view).
  
• Or you can try to learn it from examples (see Ray Mooney's talk yesterday)


• Sometimes need more than FOL
• How to handle "Many students love NLP"?
• Or "John did something yesterday" -- for this we may need second order logic, where you can quantify over relations -- "(exists R, e) R(e, John) and InThePast(e)"