Similarity, relatedness, antonymy and connotation

Read these pairs and give each one a number from 0 to 10 for how alike the two meanings are: vanish and disappear, behave and obey, belief and impression, muscle and bone, modest and flexible, hole and agreement. You probably gave the first pair close to 10, the last close to 0, and found the middle harder but not impossible. Hundreds of people did exactly this, and their averages are the numbers below.

The association column comes from the University of South Florida free-association norms: how often people answer the second word when given the first as a cue. It measures connection, not shared features.

The scores fall away smoothly. There is no point where the pairs stop being similar and start being dissimilar, which is the first thing to notice. Synonymy, from part 01, is close to a yes or no question about two senses. Word similarity is a matter of degree: two words are similar when their meanings share features, and they can share many features, a few, or none. vanish and disappear share nearly all of them. muscle and bone share a few (body tissue, anatomy) and differ on the rest. hole and agreement share essentially nothing.

The second thing to notice is that similarity is a relation between words, not senses. This is the car and bicycle idea from the end of part 01. To say whether two senses are synonyms you need a sense inventory; to ask a person how similar two words feel, you do not. That makes word similarity cheap to collect and directly comparable to anything that produces one number per word pair, which is exactly what a vector model does.

Where the numbers come from

The table is SimLex-999 (Hill, Reichart and Korhonen 2015). It contains 999 pairs: 666 noun pairs, 222 verb pairs and 111 adjective pairs, mixing concrete and abstract words. About 500 Mechanical Turk workers rated them on an integer slider from 0 to 6, and the mean ratings were then linearly rescaled to 0 to 10. The slide shows the rescaled values, which match the released data file exactly. Individual raters disagree a fair amount: the average Spearman correlation between two raters is 0.67, and between one rater and the mean of the others it is 0.78. The average is what is stable.

Why should you care about these particular numbers? Because later in this lecture they become the gold standard for intrinsic evaluation. You compute the cosine between the two vectors of every SimLex pair, rank the pairs by cosine, and report the Spearman rank correlation with the human ranking. A model that agrees with people about which pairs are more alike scores high.

Compare two pairs: coffee and tea, then coffee and cup. Coffee and tea are both hot drinks made by steeping or brewing a plant, both contain caffeine, both are served in the morning. They share features, so they aresimilar. Coffee and cup share practically no features: one is a drink from a plant, the other is a manufactured container. Yet nobody would call them unconnected. They take part together in one everyday event, drinking coffee from a cup.

That second kind of connection is word relatedness, also called association. Two words are related when they are connected in any way at all: by shared features, by a shared event, by part and whole (car and wheel), by function (pencil and paper), even by opposition (hot and cold). Budanitsky and Hirst put the hierarchy plainly: similarity is a special case of relatedness. Every similar pair is related, but many related pairs are not similar. They quote Resnik's example: cars and gasoline are more closely related than cars and bicycles, but the latter pair are certainly more similar.

The SimLex file records both quantities, so you can see the split in real numbers. clothes and closet have similarity 3.27 but association 4.83: related more than they are alike. car and bicycle have similarity 3.47 and association only 0.41: alike more than they are associated.

Semantic fields

When you collect all the words that keep turning up in the same scene, you get a semantic field: a set of words that cover one domain and bear structured relations to each other. The hospital field holds surgeon, scalpel, nurse, anaesthetic and hospital. The restaurant field holds waiter, menu, plate, food and chef. The house field holds door, roof, kitchen, family and bed. Inside a field the relations are varied: a surgeon uses a scalpel, a nurse assists a surgeon, the anaesthetic is given in the hospital. What unites them is the domain, not shared features.

Fields are not only a linguist's classification. Topic models such as Latent Dirichlet Allocation read a large collection of documents with no labels and discover clusters of words that tend to occur together, and the clusters they find look very much like semantic fields. That is a first hint of the theme of this whole lecture: the company a word keeps carries information about its meaning.

Take hot and cold. Both are adjectives. Both describe temperature. Both slot into the same frames: hot coffee and cold coffee, hot weather and cold weather, it is too hot today and it is too cold today. Line up everything you know about the two words and they agree on almost every point. They disagree on exactly one: which end of the temperature scale they name.

That is the definition of antonymy. Antonyms are senses that are opposite with respect to only one feature of meaning and otherwise very similar. It sounds paradoxical that opposites are mostly alike, but you cannot be opposite to something unless you are first comparable to it. Hot is not the opposite of Tuesday.

Kinds of opposition

The single opposed feature can be of different types. In the first group the two words name the two values of a binary choice or the two ends of a scale: long and short, fast and slow, big and little. In the second group, the reversives, the two words describe change or movement in opposite directions: rise and fall, up and down. Mohammad, Dorr, Hirst and Turney refine this further (antipodals, complementaries, gradable opposites) and note that many contrasting pairs, such as warm and cold, are not strict opposites at all.

What humans say, and why models struggle

Because antonyms differ on a feature people care about, human raters call them dissimilar. Because they share everything else, they are among the most strongly associated pairs in the language. SimLex records both, and the gap is striking. Hill and colleagues conclude that antonyms are the most strongly associated word pairs among the finer-grained relations they examined.

Now look ahead. The distributional hypothesis that drives the rest of this lecture says that words in similar contexts have similar meanings. hot and cold occur in almost identical contexts, so a model built on contexts will put them close together. Opposites even co-occur in the same sentence more often than chance would predict (Charles and Miller, cited by Mohammad and colleagues), which pulls them closer still. SLP3 is blunt about the result: automatically distinguishing synonyms from antonyms can be difficult.

A museum shop sells a replica of an ancient vase. A street stall sells a knockoff. Both objects are copies of a real thing, and a description of either would read much the same. But the first word is close to praise and the second is an accusation.

The part of meaning that differs here is connotation: the aspects of a word's meaning tied to a writer's or reader's emotions, sentiment, opinions or evaluations. Some words exist mainly to evaluate: great and love are positive, terrible and hate are negative. Others, like replica and knockoff, describe the same thing while carrying different attitudes toward it. Positive or negative evaluation in language is called sentiment, and connotation is what sentiment analysis, stance detection, and NLP work on political language and consumer reviews all exploit.

Connotation can be measured. Affect lexicons give each word a score, and the one we meet in the next concept, the NRC VAD Lexicon, gives a valence (pleasantness) between 0 and 1. Here is the SLP3 example worked through with its real numbers.

Compare napping and toxic. napping is pleasant and calm, and it puts you in no particular position of control. toxic is unpleasant and agitating. A single positive or negative score would capture the first difference, but not the second, and not the question of who has the power.

Osgood and colleagues (1957) found that people's ratings of words consistently varied along three affective dimensions, now called valence, arousal and dominance:

• Valence is the pleasantness of the stimulus. napping 0.765, toxic 0.008.
• Arousal is the intensity of emotion the stimulus provokes. napping 0.046, toxic 0.885.
• Dominance is the degree of control the stimulus exerts. napping 0.306, toxic 0.492.

Three numbers per word means each word becomes a point in a three-dimensional space. SLP3 calls this the first expression of the idea behind vector semantics: on the 1 to 9 scales of Warriner and colleagues (2013), heartbreak sits at [2.45, 5.65, 3.58]. Part 03 takes that idea and scales it from three hand-chosen dimensions to hundreds learned from text.

How the NRC VAD Lexicon was built

The numbers come from the NRC VAD Lexicon (Mohammad 2018). Version 1 covers about 20,000 English words; the released file has 19,971 entries. Asking people to rate a word on a slider is unreliable, because everyone uses the slider differently. Instead Mohammad used best-worst scaling. An annotator sees four words and picks the one highest on the dimension (say, most pleasant) and the one lowest. Over many such four-word sets, each word is scored by how often it won minus how often it lost.

Comparative judgements are much more consistent than absolute ones. Splitting the annotators into two random halves and correlating the scores each half produces gives split-half reliability of r = 0.95 for valence, 0.90 for arousal and 0.91 for dominance. Version 2, released in March 2025, extends the lexicon to over 55,000 terms (about 10,000 of them multiword phrases) on a -1 to 1 scale, with split-half Spearman of 0.98, 0.97 and 0.96.

Step back and look at what we now have. One word can map to many senses: mouse is a rodent or a pointing device. One sense can map to many words: couch and sofa. The mapping between words and concepts is many to many, and on top of it sits a set of relations, some between senses and some between whole words.

A lemma groups its senses, and polysemy is the fact that it has several. Synonymy and antonymy are relations between senses. Similarity, relatedness and connotation are graded and can be asked of whole words, which is what makes them measurable with ratings and lexicons.

This table is a list of desiderata. Any representation of word meaning we build next should put similar words near each other, keep related words in recognisable neighbourhoods, encode affect in some consistent direction, and ideally tell synonyms from antonyms. Part 03 introduces vectors as that representation. A vector space captures relatedness readily and affect reasonably well; separating true similarity from relatedness is harder, and telling synonyms from antonyms is where it struggles most, as concepts 2 and 3 warned.