Applications of NLP

Take the definition of NLP from part 01, paste it into Google Translate, and pick Arabic. In under a second the right pane fills with معالجة اللغة الطبيعية هي فرع من فروع الذكاء الاصطناعي and a Latin transliteration line underneath. Nobody touched that output. The system read 142 characters of English and produced finished Arabic. That is machine translation in its fully automatic mode, and it is the mode most people picture when they hear the words.

Now look at the second screenshot on the same slide, which looks nothing like a consumer product. The source box holds an Arabic news sentence about the Lebanese president being subjected to a fierce campaign in parliament. The translator has typed one word into the translation box, lebanese, and the system has answered with a ranked list of what the next word could be: president, suffered, exposed, president emile, before, presented, offer. The human is still writing the sentence. The machine is predicting where the sentence goes next. This is computer-aided translation, and its existence is the most important fact on the slide: the professional tool keeps the human in charge.

Two modes, and who owns the final text

Jurafsky and Martin list three uses of MT: information access, aiding human translators, and in-the-moment communication. Fully automatic MT serves information access: you want to read a page written in a language you do not know, and a rough but instant translation is worth more than a perfect one next week. Google Translate alone, they note, translates hundreds of billions of words a day between over 100 languages. The second use is aiding human translators. Here the system is not the author. It drafts a translation that a person then fixes, a workflow called post-editing. Interactive systems (Barrachina et al., 2009) go further: they sit inside the editor and predict as the person types, which is what the slide shows.

The interactive kind has a precise description. Barrachina and colleagues put it this way: in each iteration, a prefix of the translation is validated (accepted or amended) by the human and the system computes its best, or n-best, translation suffix to complete that prefix. Read the slide's list with that in mind and it stops being a random menu. The list is an n-best set. "suffered" and "exposed" are two English readings of the same Arabic verb تعرض, and "president" versus "president emile" are two choices about how much of the source to consume in one step. The translator resolves in one click what a fully automatic system would have to gamble on.

Why keep the human at all, when the top screenshot is so good? Because the same paper opens with the plain admission that current MT systems are still not perfect and that in practice their output needs to be edited to correct errors. Which errors matter depends on the text. A gisted news article can survive a wrong verb. A contract, a drug label or a court transcript cannot, and someone must be accountable for the register, the terminology and the liability. The assisted mode is how the field turned an imperfect technology into a productivity tool rather than waiting for a perfect one.

The product numbers, and why they are pipelines

Slide 8 adds two products with feature lists. The Google Translate app page counts how many languages each mode supports, and the Skype Translator blurb promises voice calls in seven languages and adds a line worth remembering: it uses machine learning, so the more you use it, the better it gets. That sentence is the whole of part 03 in miniature. The system is not a fixed set of rules; it is a model that improves with data.

The more durable lesson on the slide is structural. Typing 103 languages is one system: text in, text out. Camera mode is not. To translate the Russian street sign in the photo, the phone must first read the pixels as characters (optical character recognition), then translate the characters, then paint the English back onto the image. Conversation mode is longer still: recognise speech, translate, synthesise speech in the other language. Jurafsky and Martin describe exactly this image-centric case, OCR of the text on a phone camera image used as input to an MT system to translate menus or street signs.

Keep this picture. When part 03 introduces the linguistic pipeline from paragraphs down to morphemes, and when part 04 lists the ways language defeats a system, both are describing links in chains like these. A product that looks like one feature is usually three or four NLP tasks in a row, and the quality the user sees is the product of all of them.

Type how heavy is our planet into a search engine and two very different things can come back. On slide 9 the engine answers with a card: Earth, Mass, 5.972 × 10²⁴ kg, a photograph, and a row of related planets. On slide 10 the same query produces an AI Overview: a paragraph explaining that Earth has a mass of approximately 5.972 × 10²⁴ kilograms, about 13.1 septillion pounds, measured using gravity rather than scales, with a heading on how scientists measure it and two cited sources in a side rail. Both are question answering. They are built on opposite principles.

What the answer box has to get right

The card looks trivial and hides three separate decisions. The slide's own label names two of them: information extraction and word sense disambiguation. Start with the word "heavy". It has several senses: heavy rain, heavy traffic, a heavy heart, a heavy object. Only one of them is a physical quantity a database could store, so the system must choose the sense "has a large mass" before it can look anything up. That choice is word sense disambiguation, and the answer card's label "Mass" is the visible trace of it.

The second decision is about "our planet". No table anywhere has a row called "our planet". The phrase is a string; the answer requires a thing. Google's Knowledge Graph was announced in 2012 under exactly that slogan, things not strings: a model that understands real-world entities and their relationships to one another, so that "Taj Mahal" can mean the monument or the musician. Mapping the mention "our planet" to the entity Earth is entity linking. (The literature calls this entity linking; entity resolution more often means merging duplicate records.) Only once both decisions are made does the third step become a lookup: fetch the mass attribute of the Earth node.

Where did the graph get that number? From text, by information extraction: Jurafsky and Martin define IE as turning the unstructured information embedded in texts into structured data, for example for populating a relational database, and note that knowledge graphs of structured relational knowledge are a useful way for search engines to present information to users. The answer box is therefore the last inch of a long chain: extractors read encyclopedias and fact sheets once, store facts as graph edges, and the search engine reads the edge at query time. The same machinery returns in the email example two concepts from now.

The ratios check out from the slide's own numbers: 6.39 / 59.72 = 0.107, 4.867 / 5.972 = 0.815 and 1898 / 5.972 = 317.8. The Mars row also carries a quiet lesson: the card shows 6.39 × 10²³ kg while NASA's current fact sheet lists 6.4169 × 10²³ kg, about 0.4 percent higher. Nothing in the card is broken. An extracted fact is only as accurate as the source it was extracted from, and that is the answer box's characteristic failure: it can only be as wrong as its stored fact, and it has no way to be more right.

What changes when a language model writes the answer

Slide 10 is the same query on a search page with an AI Mode tab and an AI Overview. Nothing was looked up in a graph and rendered as a card. Instead the engine retrieved documents, handed them to a large language model, and let the model write. Jurafsky and Martin call this retrieval-augmented generation: use information retrieval techniques to retrieve documents, then use a large language model to generate an answer conditioned on those documents. The recipe is three lines. Retrieve the top passages for the query, build a prompt that contains the query and those passages, and call the model.

The screenshot shows every ingredient. The retrieved passages are the two cards in the side rail, an AIP Inside Science article titled "How Much Does Earth Weigh?" and the Wikipedia article on Earth. The generation is the paragraph, which is not a quotation from either page but a new text stitched from both, complete with a unit conversion: 5.972 × 10²⁴ kg times 2.20462 lb/kg is 1.317 × 10²⁵ lb, which is the "13,170,000,000,000,000,000,000,000 lbs" in the overview and the "13 thousand, 170 billion trillion pounds" in the AIP article. The citation chips are the last ingredient, what the textbook calls a knowledge citation that helps the user decide whether to trust the generation.

Look closely at the second chip and you can see the seams. Its snippet reads "Table_title: Earth Table_content: header: | Designations | row: ...", which is not prose at all; the retriever pulled a Wikipedia infobox as a flattened table and the model wrote around it. That is the point of the shift: the classic box extracts one structured fact and shows it, the overview retrieves raw text and composes. The output is more helpful, more explanatory and more flexible, and it introduces a failure that the card never had.

That new failure has a name. Large language models hallucinate, and a hallucination, in Jurafsky and Martin's words, is a response that is not faithful to the facts of the world. They report a study in which, asked questions about the legal domain, large language models hallucinated from 69 to 88 percent of the time, and add the more troubling half: it is not always possible to tell when a model is hallucinating, partly because LLMs are not well calibrated, so they can give a very wrong answer with complete certainty. The card can be stale. The paragraph can be wrong while sounding exactly like the truth, and the citations can point to a page that does not say what the paragraph claims.

Slide 11 shows spelling and grammar correction at work: a Word window with one sentence in it, This is bigger then that ofcourse. Two words are underlined, and the colours are not decoration. ofcourse carries a red squiggle. then carries a blue one, and the context menu under it offers a single replacement, than. Microsoft's documentation spells out the convention: potential spelling errors get a red squiggly line, potential grammatical errors get a blue one. The two colours mark two different problems, and the second is much harder than the first.

Jurafsky and Martin draw the line precisely. Non-word spelling correction is the detection and correction of spelling errors that result in non-words, like graffe for giraffe. Real-word spelling correction is the task of detecting and correcting spelling errors even if they accidentally result in an actual word of English. ofcourse is the first kind: it is not in any dictionary, so the check is a lookup that needs no context whatsoever. then is the second kind. It is a perfectly good English word, in the dictionary, spelled correctly. Nothing about the word itself is wrong. What is wrong is the word in this position, between "bigger" and "that", where English demands the comparative partner "than".

That difference is why the second problem is hard. The textbook says it directly: real-word spelling error detection is a much more difficult task, since any word in the input text could be an error. And it is not a rare case. Studies suggest that between 25 and 40 percent of spelling errors are valid English words. A checker that only consults a dictionary silently passes a quarter to nearly half of all mistakes. To catch them, the system must know what English sentences look like, which is to say it needs alanguage model, a model that assigns higher probability to "bigger than that" than to "bigger then that", or a grammar rule that encodes the same preference by hand.

The noisy channel, in one line

The classical machinery for both kinds is the noisy channel model. Imagine the writer meant some word w and a noisy channel, their fingers, produced x. The corrector wants the intended word that best explains what it sees.

The candidate set V is kept small because the majority of spelling errors consist of a single-letter change, so candidates within one insertion, deletion, substitution or transposition of the typed string (Damerau-Levenshtein distance 1) cover most cases. For a non-word, the language model term just prefers common words. For a real-word error, the channel model works against the correction: in the Mays et al. model the textbook uses, P(x | w) is alpha (about 0.95) when the typed word is the intended one, and only the small remainder 1 - alpha is spread over candidates such as than. The language model term P(w) must therefore outweigh that bias, and it can only do so from context.

This is also the oldest NLP application most people use every day. Jurafsky and Martin note that spelling correctors are ubiquitous in pretty much any software that relies on keyboard input, from word processors to search boxes to phone keyboards, and the same noisy channel idea reappears in speech recognition and machine translation. Real-word correction was also an early case of a language model doing linguistic work that no dictionary could, which is the pattern the whole modern field follows.

An email arrives for Dan Jurafsky, dated January 15, 2012, subject "curriculum meeting". The body is two lines: "Hi Dan, we've now scheduled the curriculum meeting. It will be in Gates 159 tomorrow from 10:00-11:30. -Chris". Beside it sits a grey card that the mail client has filled in by itself: Event, Date, Start, End, Where, and a blue button, Create new Calendar entry. The email is prose. The card is a record. Getting from one to the other is information extraction, and the card is the clearest picture of it in the whole lecture.

The card is an instance of what Jurafsky and Martin call template filling: find documents that invoke a particular script, then fill the slots of the associated template with fillers extracted from the text. The script here is "a meeting is being scheduled", the template has five slots, and the textbook adds the distinction that makes the slide interesting. Slot fillers may consist of text segments extracted directly from the text, or they may consist of concepts that have been inferred from text elements through some additional processing. Sort the five fields by that rule and they split cleanly in two.

Resolving "tomorrow"

The Date field is the one worth slowing down on, because the word "tomorrow" contains no date at all. Jurafsky and Martin call this temporal normalization, the task of mapping a temporal expression to a point in time or to a duration, and they explain why it is possible: most temporal expressions are incomplete and only implicitly anchored, often with respect to the dateline of the article, which they call the document's temporal anchor. The values of expressions such as today, yesterday or tomorrow can all be computed relative to that anchor, and the rule for these two is the simplest one there is. Tomorrow adds a day to the anchor, yesterday subtracts one.

Two threads tie this concept to its neighbours. First, the textbook's own exercise on this chapter asks for exactly the application on the slide: associating temporal expressions connected with events in email with specific calendar entries, which is a sign of how central the example is. Second, the graph that the answer box read from in the previous concept was populated by the same kind of extractor working over encyclopedias instead of inboxes. Search returns documents. Extraction returns fields with normalised values that a program can act on, and the blue button is the program acting.

Slide 13 puts a camera next to a list of the things people talk about when they review it: zoom, affordability, size and weight, flash, ease of use. Under "Size and weight" it collects three remarks: "nice and compact to carry!", "since the camera is small and light, I won't need to carry around those heavy, bulky professional cameras either!", and "the camera feels flimsy, is plastic and very light in weight you have to be very delicate in the handling of this camera". Read the second and third again. Both say the camera is light. Decide below whether each one is praise or complaint, then read on.

The slide title pairs information extraction with sentiment analysis, and that pairing is the lesson. Bing Liu, whose book defined the field, states the principle: an opinion consists of a sentiment (positive or negative) and a target, and an opinion without its target being identified is of limited use. He calls the finer-grained task aspect level analysis, earlier called feature-based opinion mining after Hu and Liu's 2004 work on exactly this kind of camera review, and he formalises an opinion as a tuple of entity, aspect, sentiment, holder and time. Then he adds the line that explains the slide title: that tuple is basically a database schema. Sentiment analysis at this level is an extraction task first. Find the aspect, then judge the polarity of what was said about it.

Now the word "light" can be explained rather than just noticed. In the second review it sits next to "small" and is set against "heavy, bulky professional cameras", so it means easy to carry. In the third it sits next to "flimsy", "plastic" and "delicate", so it means insubstantial. The physical fact is identical. The polarity is a property of the aspect and the context together, not of the word and not of the document. Liu's own camera example makes the mirror image point: "my wife thinks it is too heavy for her" is a negative opinion about weight, and nothing about the word "heavy" on its own tells you that.

Aspects are not always spelled out either. Hu and Liu give the sentence "While light, it will not easily fit in pockets" and point out that the word size does not appear in it, so the system has to infer the aspect before it can score the opinion. Their illustrative summary for a digital camera counts opinions per feature, which is what the grey bars beside the slide's review list are sketching.

1. Find the sentences that express an opinion.
2. Extract the aspect each opinion targets, explicit (flash) or implicit (light, therefore size and weight).
3. Classify the polarity of the opinion about that aspect, using the surrounding words.
4. Aggregate into per-aspect counts, so a reader sees two positive and one negative on size and weight.

The order matters. A whole-document score would average the third review's complaint about build against whatever it praised elsewhere, perhaps affordability, and report a lukewarm three stars that helps nobody. Liu makes the same point with a restaurant: "although the service is not that great, I still love this restaurant" is positive about the restaurant and negative about the service, and a document-level system does not discover what exactly people liked and did not like.