Majid Al-RaimiPhD notes / KFUPM

ICS 582/Lecture 02Words and tokens

Overview
Full guide
Glossary
Reference
Practice quiz

Parts11

What counts as a word4 concepts, slides 1-10
Zipf and Heaps: the shape of vocabulary4 concepts, slides 11-16
Morphemes: the parts of words3 concepts, slides 17-21
Clitics and morphological typology3 concepts, slides 22-28
Corpora, Unicode and UTF-84 concepts, slides 29-37
Tokenization units and byte-pair encoding5 concepts, slides 38-48
Tokenizers in practice and sentence segmentation4 concepts, slides 49-55
Regular expressions: the core syntax4 concepts, slides 56-65
Regular expressions in practice6 concepts, slides 66-78
Minimum edit distance: alignments and search4 concepts, slides 79-86
Filling the DP table and recovering the alignment5 concepts, slides 87-94

Majid Al-RaimiPractice quiz

ICS 582Lecture 02Practice quiz

Practice quiz

There are no practice questions for this lecture yet.