Clitics and morphological typology

Start with an English sentence that Zwicky and Pullum (1983) use in section 2 of their paper: The person I was talking to's going to be angry. The 's is a reduced is, and it is hanging off the preposition to. Now vary the host: The ball you hit's just broken my window puts it on a verb, Any answer not entirely right's going to be marked on an adjective, The drive home tonight's been really easy on an adverb. One element, four host categories. It is a reduced auxiliary throughout, is in the first and third sentence and has in the second and fourth, and in every case it does exactly what the separate word is or has would do.

Compare the plural -s of cats. It attaches to nouns and to nothing else, and it never does the job of a word: there is no sentence in which plural -s is the verb or the subject. That difference in behavior is what separates a Clitic from an Affix. Jurafsky and Martin define a clitic as "a morpheme that acts syntactically like a word but is reduced in form and attached (phonologically and sometimes orthographically) to another word" (SLP3, chapter 2). The two halves of that definition pull in opposite directions, which is why clitics are confusing: in the syntax they are words, in the sound stream they are part of a word.

Why spelling is not the test

English happens to mark most clitics with an apostrophe (I'm, we've, possessive 's), and French does the same with l' in l'opéra. That is a writing convention, not a grammatical fact. Arabic writes every clitic solidly attached with no separator at all, so if attachment in writing were the criterion, Arabic would have no clitics and English would have almost none outside contractions. The slide's key sentence is worth memorizing verbatim: attachment in writing is not enough, use diagnostics.

The diagnostics come from Zwicky and Pullum. Their first criterion is the one this course uses as the primary test, and they state it as: "Clitics can exhibit a low degree of selection with respect to their hosts, while affixes exhibit a high degree of selection with respect to their stems." This is Selectivity. In a footnote they give it as a ladder: an element that attaches to any word of a major class is least selective, then one that attaches only to verbs, then only to a subclass of verbs, then only to a few specified verbs. The further down the ladder, the more affix-like the element. The second test the slide names is paradigm membership: an affix fills one cell of a table of forms, a clitic does not. All six of their criteria are in the table below alongside the slide's two tests, and you should be able to name at least selectivity, arbitrary gaps and idiosyncrasy.

Three Arabic cases and a fourth that refuses to settle

Arabic is where the test earns its keep, because the writing system gives you no hint. Take وَبِالْبَيْت, wa-bi-al-bayt, roughly "and in the house". The wa- is the conjunction "and" and the bi- is the preposition "with, by, in". Both are function words in the syntax, and both attach to whatever word begins the phrase: a noun here, a verb in wa-kataba, a pronoun, a particle. Lowest rung of the selectivity ladder, no paradigm. They are proclitics, clitics that lean forward onto the following word.

Now كِتَابُهُ, kitāb-u-hu, "his book". The -u is the nominative case vowel, an inflectional ending. The -hu is the pronoun "he" in possessive position, and it is the possessor of the noun in the syntax. It attaches to nouns (kitāb-u-hu), to verbs as an object (kataba-hu, "he wrote it") and to prepositions (بِهِ, bi-hi, "with it"). Three host categories, a syntactic role of its own: an Enclitic, leaning backward onto the preceding word.

Finally كَتَبْتُ, katab-tu, "I wrote". The -tu also means something pronoun-like (first person singular), which is exactly what tempts students to call it a clitic. Apply the diagnostics. Selectivity: -tu attaches to verb stems and to nothing else. Paradigm: it is one cell of the perfective conjugation, next to -ta (you, masculine), -ti (you, feminine), -a (he) and -at (she). Syntactic role: it is subject agreement on the verb, and a separate subject pronoun can still appear alongside it. Every test says Inflection, so -tu is an inflectional affix on the Stem katab.

The fourth row is the one the slide flags as borderline, and it is worth understanding why rather than memorizing the label. The article al- is a function element like French l', which argues clitic. But it attaches only to nouns and adjectives, and it alternates with the indefinite ending -n (bayt-un versus al-bayt-u) inside a definiteness system, which argues prefix. Arabic NLP tools split the difference. Pasha et al. (2014) list two segmentation schemes for MADAMIRA: ATB, which "tokenizes all clitics except for the definite article", and D3, in which "basically all clitics are tokenized". ATB also leaves the future particle sa- attached to the verb, where D3 splits it off. Alotaiby et al. (2010) note that their parse-it-as-a-separate-word test "covers most of the clitics except the definite article". When the tools disagree, borderline is the honest verdict.

Clitic status and tokenization policy are different questions

The slide ends on a practical fork: keep the clitic attached, or split it? Linguistics does not answer that Tokenization question for you, and the cleanest proof is English n't. Zwicky and Pullum devote their paper to showing that n't passes all six criteria as an inflectional affix: it attaches only to auxiliaries, it has arbitrary gaps (there is no amn't), it has idiosyncratic forms (won't, shan't), and it cannot attach to a host that already carries a clitic (I'd've is fine, I'dn't is not). Yet the Penn Treebank tokenization standard splits it, producing does n't (SLP3, section 2.8). The tokenizer made a modeling decision (a negation token is useful) that contradicts the linguistic analysis, and it was right to.

Modern subword tokenizers make the same kind of decision by frequency rather than by rule. Jurafsky and Martin observe that the GPT-4o tokenizer segments 's off proper nouns such as Jane but keeps it inside frequent words such as she's (SLP3, section 2.4.3). For Arabic the choice is explicit and named: whitespace, ATB or D3. Try the three on the same two strings.

Take four languages and watch how each packages grammar. English and Turkish can share one meaning, "from my houses". English spends three words: from, my, house-s, four morphemes spread over three words. Turkish spends one: ev-ler-im-den, house-PL-1SG.POSS-ABL, four morphemes in one word, each suffix doing exactly one job in a fixed order. Spanish habl-o, "I speak", is a different meaning chosen to show a different trick: the single vowel -o carries person, number, tense and mood all at once, two morphemes carrying four features, and you cannot cut it into a piece for each. Mandarin wǒmen míngtiān qù Běijīng, "we tomorrow go Beijing", has almost no Inflection (the plural -men of wǒmen is about as far as it goes); word order and free words carry the grammar.

Turkish, Spanish and Mandarin illustrate three of the four labels on slide 24 (the fourth, polysynthetic, comes below), and English is the analytic baseline. The labels line up on two independent axes rather than one scale. The first axis is how many morphemes a word carries, from isolating (about one) to polysynthetic (a word can hold what English needs a clause for). The second axis is how cleanly the morphemes can be cut apart, from agglutinative (one morpheme, one feature, clean seams) to fusional (one segment bundles several features). This two-axis picture goes back to Sapir's 1921 classification, which separated degree of synthesis from technique of combination (Arkadiev 2019, and SLP3 section 2.2 uses the same two ideas).

The Turkish and Spanish rows deserve a closer look, because the difference between them is the second axis. Bickel and Nichols (WALS chapter 21) call Turkish case and number monoexponential: -ler means plural and nothing else, and the case suffixes -in, -i, -e each mean one case. In a fusional language such as Russian, they note, there is "no element that exclusively expresses case or number": the genitive plural ending says both at once. English has the same fusion in miniature. Jurafsky and Martin point out that the -s of she reads encodes third person, singular and present tense in one segment, exactly as Spanish -o does.

The polysynthetic row borrows its form from a neighboring language on purpose. The slide names Inuktitut but prints no word, warning that examples vary by dialect and orthography and that the key property is morpheme density. Jurafsky and Martin's example is Koryak: t-ə-nk'e-mejŋ-ə-jetemə-nni-k, one word meaning "I sewed a lot of yurt covers in the middle of a night" (SLP3, example 2.9), and that is the form to quote for the polysynthetic type. Gupta and Boulianne (2020) describe Inuktitut as showing "a much higher degree of polysynthesis than other agglutinative languages usually considered in ASR, such as Finnish or Turkish", which is a useful reminder that polysynthetic languages are usually agglutinative too: the axes are independent.

Tendencies, not boxes

The slide's mental model is that a language can be "more X" in one subsystem and "more Y" in another, and this is not a hedge. Jurafsky and Martin write that "languages can make use of different morphological systems so it would be more accurate to talk about these as general tendencies." Bickel and Nichols (WALS chapter 22) give an example inside one language: "the English past, for example, is more synthetic than the future", because walked packs tense into the verb while will walk spends a separate auxiliary. So when you classify, name the subsystem: English verbs are mostly analytic with a fusional pocket, English Derivation is productive and concatenative, and Arabic verbs are fusional while Arabic attached function elements are clitic-like.

Where English and Arabic sit

Slide 26 places the two languages this course cares about. English is largely analytic: auxiliaries and word order do most of the grammatical work, and inflection is limited to a handful of endings, -s, -ed, -ing. But it has fusional pockets. am, is and are each bundle person, number and tense in a form with no separable ending at all, and went replaces the expected go-ed wholesale, a case of Suppletion. Where English is genuinely productive is derivation and Compounding: un-happi-ness, laptop, and every new noun coined this year.

Modern Standard Arabic sits elsewhere on both axes. Word formation follows templatic morphology, so a consonantal Root such as K-T-B is interleaved with vowel patterns, a non-concatenative mechanism that no amount of prefix and suffix cutting will recover. The verb system is largely fusional, and it is large: Obeid et al. (2020) note that this richness "leads to MSA verbs with upwards of 5,400 forms". On top of the templatic core sit the concatenative pieces from the previous concept, proclitics wa-, bi-, li- and the pronominal enclitics, which behave as clitics in NLP, plus the borderline al-. Sapir's own table, as reproduced by Arkadiev, files English as fusional and analytic and Arabic as fusional, symbolic and synthetic, which is the same placement in older words.

The typology labels can be turned into a single number. Take a long text, count its morphemes, count its words, and divide. Joseph Greenberg proposed this ratio in 1960 as the Index of synthesis, "calculated as the ratio of morphemes to words in a sufficiently long text" (Arkadiev 2019). Try it on four English words: The cats walked home is the (1), cat-s (2), walk-ed (2), home (1), six morphemes over four words, an index of 1.5. Turkish ev-ler-im-den alone is 4 / 1 = 4.0, and the English equivalent from my houses is 4 / 3 ≈ 1.33.

Greenberg computed the index for eight languages, and the result is the number line on slide 27, reproduced from the same figure in Jurafsky and Martin (SLP3, figure 2.3). Vietnamese sits at the bottom with almost exactly one morpheme per word. English is close behind, which confirms the analytic label of the previous concept. Old English, Yakut, Swahili and Sanskrit cluster in the middle band, and Greenlandic (Eskimo in Greenberg's paper) stands far to the right at almost four morphemes per word. The bands drawn through this line are the ones you already know: analytic below about 2, synthetic between 2 and 3, polysynthetic above 3.

Greenberg's paper contains several more indices, and two of them explain what the synthesis number hides. Eskimo scores 2.72 on suffixation and 1.75 on inflection against English's 0.64 and 0.53; Swahili scores 1.16 on prefixation against English's 0.04. Swahili sits mid-synthetic not because it stacks suffixes but because its noun-class and agreement system lives in prefixes. Same band, different subsystem, which is the tendencies-not-boxes lesson in numbers.

Why the index decides your tokenizer

Slide 28 closes the morphology chapter with one sentence: "word" is not universal, because morphology and clitics change what counts as a Token. The index of synthesis makes that quantitative. Every extra morpheme per word multiplies the number of distinct surface forms a Lemma can take, and every clitic that attaches multiplies it again. Since a model's Vocabulary is a set of surface forms, a language with a high index and heavy cliticization will have a higher Type-token ratio (TTR), far more types for the same number of tokens, and far more of them will be Out-of-vocabulary (OOV) at test time. Recall Heaps' law from part 02: rich morphology tends to raise the exponent, so the vocabulary keeps growing quickly as the corpus grows.

At the polysynthetic end the effect is far larger. Schwartz et al. (2020) compare the parallel Inuktitut and English sides of the Nunavut Hansard. The numbers below come from their table 3.2.

Inuktitut produces about 26 times as many types from about half as many tokens. Gupta and Boulianne (2020) report the practical consequence for speech recognition: even with a 1.3 million word vocabulary, held-out Inuktitut stories still have "more than 60% of words out-of-vocabulary". A word-level vocabulary cannot be made large enough. That is the open-vocabulary problem, and slide 28 lists three responses to it.

1. Subword tokenization with BPE or WordPiece. Sennrich, Haddow and Birch (2016) motivated BPE for translation precisely because vocabularies are "typically limited to 30 000 to 50 000 words, but translation is an open-vocabulary problem", "especially for languages with productive word formation processes such as agglutination and compounding". WordPiece, from Wu et al. (2016), divides words "into a limited set of common sub-word units". Part 06 builds BPE from scratch.
2. Morphological segmentation when linguistic structure matters. A frequency-driven tokenizer will not reliably cut وَبِالْبَيْت at the clitic boundaries. MADAMIRA and CAMeL Tools do, using an analyzer and an explicit scheme (ATB or D3), which is what you want when a downstream task needs the bare lemma.
3. Careful preprocessing: case, punctuation and a stated clitic splitting policy. Jurafsky and Martin describe the pre-tokenization stage as "breaking the input at spaces and punctuation, stripping off clitics, and breaking numbers into sets of digits" (SLP3), which is the tokenization pipeline's first pass.

The last thread to pull runs into part 07. A multilingual subword vocabulary is trained mostly on English, and Jurafsky and Martin note that such tokenizers "use most of the tokens for English" while "the other languages tend to get their words split up into shorter tokens" (SLP3, section 2.4.3). Arabic suffers twice: it has less training data in the mix, and its clitics and templatic verbs (upwards of 5,400 forms per verb) spread that data across far more surface forms, so fewer of them earn a merge. The same sentence costs more tokens in Arabic than in English, which means more compute per sentence and a shorter effective context.