Introduction to natural language processing

AOverview

This lecture draws the map that every later lecture in the course fills in. It starts with a wall of company logos and a one-sentence definition, and it ends with four ordinary words, I made her duck, that carry five meanings. Between the two lies the whole shape of natural language processing: what the field is for, what it ships, how it is built, and why language refuses to make the building easy.

As a PhD student you need it for three reasons. The exam questions on this material are definitional and list-shaped: the definition and its three goals, the three names of the field, the five AI subfields, the four approaches, the levels of linguistic knowledge from phonetics to discourse, the five readings of the duck sentence, the six sources of difficulty and the three tools for progress. Your research will have to state, in the introduction of any paper or thesis chapter, which level a task lives at and whether the knowledge at that level is coded by hand or learned from data, and if that research touches Arabic it inherits the difficulties this lecture names (diglossia, clitics, no capital letters). And any real system you build or review is a chain of components in which every link resolves an ambiguity and every link can be wrong.

The path has four stops. You define NLP by its goal rather than its tool and place it among its parent disciplines and its sibling subfields. You turn seven product screenshots into five reusable ideas, from human-in-the-loop translation to aspect-level sentiment. You learn the one axis that separates the four approaches (who writes the knowledge, a human or the data) and the levels of linguistic knowledge that any approach must fill. Finally you watch one sentence fork into five readings and see that most of the field is choosing one of them.

Success looks like

• Recite the definition of NLP with its three goals and three alternative names, and draw the Venn of linguistics, computer science and AI with one contribution word in each circle.
• List the five AI subfields, place deep learning under machine learning, and name one system where NLP meets vision, robotics and knowledge representation.
• Name the NLP task hiding behind any product, from the sense disambiguation inside an answer box to the template filling behind a calendar entry.
• Contrast the four approaches on one axis, what a human writes versus what the data supplies, with one strength and one weakness each.
• Order the levels of linguistic knowledge from phonetics to discourse, say which apply to speech only, and place any task at its level.
• Produce five readings of I made her duck, name the level of ambiguity behind each, and bracket both attachments of a prepositional phrase.
• List the six sources of difficulty beyond ambiguity with the subproblem each defeats, and state the three tools and three disciplines that make progress possible.

How to study this lecture

1. Choose your route. Read the full guide in one sitting for the big picture, or work through one part at a time when you want depth.
2. Answer every recall prompt in your head, or on paper, before you reveal it. The lists in this lecture only stick if you retrieve them, and the five readings of the duck sentence are worth attempting cold.
3. Take each quiz and read the explanation even when you are right. The explanations carry the distinctions an examiner probes, such as stemming against lemmatization or semantics against pragmatics.
4. Use the simulators: run a paragraph through the pipeline stepper, fill the email template, and cycle the duck sentence through its readings. Seeing a decision go wrong is faster than reading that it can.
5. Mark a concept as understood only when you could explain it to a classmate without looking. Unmarked concepts show you where to return.
6. Keep the glossary open for terms, and use the reference sheet for the lists, tables and the corrected slide details when you review.
7. Come back after a few days and retry the recall prompts and quizzes cold. Spaced practice builds the long-term memory an exam needs.

BThe 4 parts

1. 01What natural language processing isThe definition of NLP, its three goals, its other names, and where it sits at the intersection of linguistics, computer science and artificial intelligence alongside the other AI subfields.3 conceptsSlides 1-618 min
   1. 1.1A field defined by its goal, and known by three other names
   2. 1.2Three parent disciplines, one intersection
   3. 1.3Where NLP sits inside AI, and why machine learning is both sibling and tool
2. 02Applications of NLPFive application families with the products that ship them: fully automatic and computer-aided machine translation, web question answering before and after LLMs, spelling and grammar correction, information extraction, and aspect-level sentiment analysis.5 conceptsSlides 7-1330 min
   1. 2.1Machine translation: fully automatic, or a tool in a translator's hands
   2. 2.2Question answering: extracting one fact versus generating an answer with citations
   3. 2.3Spelling and grammar: non-word errors versus real-word errors
   4. 2.4Information extraction: turning an email into calendar fields
   5. 2.5Aspect-level sentiment: the same fact can be praise or complaint
3. 03How NLP is done: approaches and linguistic knowledgeThe four families of NLP approaches, the pipeline from paragraphs to morphemes, the levels of linguistic knowledge from phonetics to discourse, and the contrast between a machine and a child who is predisposed to acquire language.4 conceptsSlides 14-1924 min
   1. 3.1Four ways to build an NLP system: who writes the knowledge, the human or the data
   2. 3.2From paragraphs to morphemes: the text processing pipeline
   3. 3.3Levels of linguistic knowledge: from speech sounds to discourse
   4. 3.4The machine is not predisposed: why every level must be coded or learned
4. 04Why NLP is hard and how progress is madeAmbiguity at every level shown through 'I made her duck', the telescope sentence and real newspaper crash blossoms, six further sources of difficulty, and the knowledge sources and disciplines that make progress possible.4 conceptsSlides 20-2524 min
   1. 4.1One sentence, five meanings: where ambiguity lives
   2. 4.2Ambiguity is pervasive: attachments and crash blossoms
   3. 4.3Six more reasons language is hard
   4. 4.4Making progress: three tools and three parent disciplines

CGlossary and reference

• GlossaryEvery term from the lecture, defined once, with links back to the slides that use it.
• Reference sheetThe lecture on one printable page, for revision before a quiz or exam.