ICS 582Lecture 01Glossary
Glossary
Every term in Introduction to natural language processing, defined once and used the same way in every part. Each entry links to the slides where the idea appears.
- Terms
- 66
- Letters
- 17
A
- Ambiguity
The property of an input having more than one possible interpretation at some linguistic level, as in the at least five readings of 'I made her duck'.
- Arabizi
Arabic written in Latin letters, with digits standing in for sounds that have no Latin equivalent (3ala for على); common in chat and social media and a form of the non-standard text problem on slide 23.
- Artificial intelligence (AI)
The field whose subfields include natural language processing, knowledge representation and deduction, machine learning, computer vision and robotics; NLP is one of its branches.
- Aspect (of an opinion)
The attribute of an entity that an opinion targets, such as size and weight, zoom or flash on slide 13; aspect-level sentiment analysis extracts the aspect first and then classifies the polarity of what was said about it.
C
- Classifier
A model that assigns an input to one of a set of categories; a core computer science tool used throughout NLP.
- Clitic
A morpheme that acts syntactically like a word but is reduced in form and attached to another word, such as the English 's and n't split off by tokenization, or the Arabic prepositions and conjunctions written onto the following word.
- Computational linguistics
An alternative name for NLP that stresses the linguistics side of the field: computational models of human language and the algorithms that process it.
- Computer vision
The AI subfield on slide 6 that interprets images and video; it meets NLP in image captioning and in optical character recognition, whose output is text.
- Computer-aided translation
Using NLP to help a human translator, for example by predicting and offering ranked completions of the target sentence as the translator types.
- Coordination ambiguity
Structural ambiguity over what a conjunction such as 'and' joins, as in 'old men and women'; with attachment ambiguity, one of the two common kinds of syntactic ambiguity behind slide 21.
- Coreference resolution
Deciding which mentions in a text refer to the same entity, for example that 'they' refers back to the students; the standard task at the discourse level of slide 17.
- Crash blossom
A real newspaper headline whose compressed syntax admits an unintended reading, named after the headline 'Violinist Linked to JAL Crash Blossoms'.
D
- Deep learning
Machine learning with multi-layer neural networks that learn their own representations from raw input; listed under ML on the AI subfield chart and used for end-to-end NLP.
- Diglossia
The coexistence of a formal written variety, Modern Standard Arabic, with the spoken dialects used in everyday life; the Arabic counterpart of the non-standard text problem on slide 23, since social media carries dialect into writing.
- Discourse
The level of linguistic knowledge about units larger than a single sentence, such as how sentences in a conversation or document relate.
- Dynamic programming
A computer science technique for solving problems by combining solutions to overlapping subproblems; one of the algorithmic tools NLP draws from computer science.
E
- End-to-end NLP using deep learning
Training a single deep neural network to map raw input text or speech directly to the desired output, learning intermediate representations instead of hand-built pipeline stages.
- Entity linking
Mapping a mention in text to the real-world entity it refers to, for example reading 'our planet' on slide 9 as the knowledge graph node Earth before its mass can be looked up. The literature usually calls this entity linking; entity resolution more often means merging duplicate records.
H
- Hallucination
A large language model response that is not faithful to the facts of the world or to the retrieved sources; the characteristic failure of generated answers such as the AI Overview on slide 10, made worse because the model can state it with complete confidence.
- Human language technology
An engineering-oriented name for NLP that emphasises building deployable systems that process human language.
- Hybrid approach
Combining rule-based components with machine learning or deep learning components in one NLP system.
I
- Idiom
A multiword expression whose meaning is not composed from its parts, such as 'throw in the towel'.
- Information extraction (IE)
Turning unstructured text into structured records, for example filling event, date, start, end and location fields from an email about a meeting.
K
- Knowledge graph
A store of real-world entities and their relationships (things, not strings) populated by information extraction, which an answer box such as the one on slide 9 reads at query time to return a fact like the mass of Earth.
- Knowledge of language
The linguistic knowledge across all levels from phonetics to discourse that an NLP system must have, hand-coded or learned, to process language.
- Knowledge representation and deduction
The AI subfield on slide 6 that stores facts and relations in a form a program can reason over and draws conclusions from them; the standard name is knowledge representation and reasoning, and it meets NLP in question answering over knowledge graphs.
L
- Language model (LM)
A model that assigns probabilities to word sequences, so it can prefer 'bigger than that' over 'bigger then that'; the P(w) term of the noisy channel. An LLM is a very large one.
- Large language model (LLM)
A very large neural language model that can generate fluent text; LLM-powered question answering produces a written overview with cited sources instead of a single extracted fact.
- Lemmatization
Reducing a word to its dictionary form (lemma) using vocabulary and morphological analysis, for example 'running' to 'run'.
- Lexeme
A vocabulary item, the unit of the lexical level that sits between morphology and syntax in the linguistic representation stack.
- Lexical ambiguity
Ambiguity in a single word, either its part of speech ('duck' as noun or verb) or its sense ('make' as cook, create or cause).
M
- Machine learning (ML)
The AI subfield in which systems learn behaviour from data rather than hand-written rules; deep learning is its sub-area and the basis of most modern NLP.
- Machine learning based NLP
Building NLP systems by training statistical classifiers or sequence models on annotated data, typically with human-designed features.
- Machine translation (MT)
Automatically translating text or speech from one human language to another, either fully automatically or as an aid to human translators.
- Morphology
The level of linguistic knowledge about how words are built from meaningful parts (morphemes); the first level shared by speech and text.
N
- Named entity
A proper name of a person, place, organisation, work or gene in text; tricky cases such as 'A Bug's Life', 'Let It Be' or the 'for' gene look like ordinary words.
- Natural language processing (NLP)
A branch of artificial intelligence concerned with interactions between computers and human language, with the goal of enabling human-machine communication, improving human-human communication, or doing useful processing of text or speech.
- Neologism
A newly coined word such as 'unfriend' or 'webinar' that is absent from existing vocabularies and training data.
- Noisy channel model
The classical model for spelling correction: the writer intended a word w, a noisy channel produced the observed x, and the corrector chooses the w that maximises the channel probability P(x given w) times the language model probability P(w).
- Non-standard language
Text that departs from standard spelling and grammar, such as 'Guys plz d/l the lettr Ive written 2 Jeff Gaspin', which defeats systems trained on edited text.
- Non-word spelling error
A spelling error that produces a string not in the dictionary, such as 'ofcourse' on slide 11; it is caught by dictionary lookup with no context needed.
O
- Orthography
The writing system of a language; the text-side counterpart of phonology that leads into morphology.
P
- Parsing
Analysing the syntactic structure of a sentence, for example deciding which phrase a prepositional phrase attaches to.
- Part of speech
The grammatical category of a word (noun, verb, adjective, pronoun and so on); a word such as 'duck' on slide 20 or 'strikes' on slide 22 can belong to more than one, and part-of-speech tagging chooses between them.
- Phonetics
The level of linguistic knowledge about speech sounds themselves; the innermost ring, used only for speech input.
- Phonology
The level of linguistic knowledge about phonemes, the sound units that distinguish words in a language; sits between phonetics and morphology for speech.
- Pragmatics
The level of linguistic knowledge about meaning in context, including the speaker's intent and what is left unsaid.
- Prepositional phrase attachment
The syntactic ambiguity of which phrase a prepositional phrase modifies, shown by 'I saw the man with a telescope'.
Q
- Question answering (QA)
Producing a direct answer to a natural language question, for example answering 'how heavy is our planet' with the mass of Earth, using information extraction and word sense disambiguation.
R
- Real-word spelling error
A spelling or grammar error whose result is a valid word in the wrong place, such as 'then' for 'than' on slide 11; lookup passes it, so only context, scored by a language model or a grammar rule, can catch it.
- Retrieval-augmented generation (RAG)
Question answering that retrieves documents for the query, builds a prompt from the query and those passages, and has a large language model generate an answer conditioned on them, as the AI Overview on slide 10 does with its cited sources.
- Robotics
The AI subfield on slide 6 concerned with perceiving and acting in the physical world; it meets NLP when a robot follows natural language instructions.
- Rule-based NLP
Building NLP systems from hand-written rules, grammars and lexicons rather than from data.
S
- Segmentation
Deciding where the units of text begin and end, for example whether 'New York-New Haven' is one name or two, in tokenization or entity recognition.
- Semantics
The level of linguistic knowledge about the literal meaning of words, phrases and sentences.
- Sentence boundary disambiguation
Splitting a paragraph into sentences by deciding which punctuation marks end a sentence, since periods also appear in abbreviations and numbers.
- Sentiment analysis
Determining the opinion expressed in text, often per attribute (aspect) of a product, for example judging whether a review's remark about size and weight is positive or negative.
- Speech and language processing
An alternative name for NLP, and the title of the Jurafsky and Martin textbook, that makes explicit that spoken as well as written language is in scope.
- Spelling and grammar correction
Detecting and correcting non-word spelling errors such as 'ofcourse' and real-word grammar errors such as 'then' for 'than' using context.
- Stemming
Reducing a word to a stem by chopping affixes with rules, without guaranteeing the result is a real word.
- Syntactic ambiguity
Ambiguity in how words group into structure, such as whether 'with a telescope' attaches to the man or to the act of seeing.
T
- Temporal normalization
Mapping a temporal expression to a point in time or a duration, anchored to the document date; on slide 12 'tomorrow' in an email dated January 15, 2012 becomes Jan-16-2012.
- Tokenization
Splitting a sentence into words or tokens, the step that produces words from sentences in the linguistic pipeline.
W
- Word sense disambiguation
Choosing the intended meaning of a word with several senses from its context, for example reading 'heavy' as mass in 'how heavy is our planet'.
- World knowledge
Knowledge about the world outside language that interpretation depends on, such as knowing that 'Mary and Sue are sisters' means sisters of each other while 'are mothers' does not.