Lecture 2: NLP Applications and Some History

There are loads of applications of NLP technology; we'll talk about some in more detail at the tail end of the course.
One of the first applications was translation

• Perhaps one of the first CS problems
• Wanted to translate Russian to English during the cold war
• Primary trend: think of translation as a problem of code breaking
• This eventually led to information theory
• Claude Shannon's noisy channel model
• Shannon's notion of entropy and the Shannon game

At the same time, CS was being developed, and so formal models like automata became very popular for explaining natural languages, since they were so good at explaining formal languages.

In the 60s, there was a split into the AI camp (which was primarily symbolic) and the statistical camp (which was mostly EE folks working on speech or OCR).

In 1964, the ALPAC report killed much funding of AI research in the US.  This, together with Chomsky's anti-statistics view (1959) led to a decade where a lot of this stuff was not mainstream.

In the 70s and early 80s, the IBM folks (statisticians) pushed statistical speech recognition using hidden Markov models.  Logic arose as a language for talking about grammar (eg., LFG, which we'll see later).  NLU came about, beginning with a blocks world ("put the red block next to the green one") and then later the Yale school (Schank and colleagues) with conceptual knowledge.  This was also when lots of work was being done on discourse.

In the late 80s and early 90s, finite state methods and statistics made a comeback, led largely by the translation efforts at IBM, and the work on morphology and phonology by Kaplan and Kay, and syntax by Church.  This was the beginning of the return of empiricism.

Nowadays, we have tons of data and empiricism is at it's peak (maybe???) and there's a lot you can do when you have lots of data.  We'll see some examples in class for several of the linguistic phenomena we talked about last time.

Other big applications:

• Automatically summarizing documents, or collections of documents
• Web search (only some people consider this true NLP)
• Question answering
• Dialogue systems
• Speech recognition
• Natural language generation
• Sentiment/opinion analysis
• ... lots more ...