Edit distance fills a dynamic-programming table for two sequences: predict one cell, then follow the cheapest insert, delete, or substitute path.

Edit Distance

See why add-one smoothing is only the first repair, then predict how Kneser-Ney uses continuation contexts to allocate backoff mass.

Smoothing: Laplace and Kneser-Ney

N-gram language models turn local sequence counts into next-token probabilities, exposing both the power of count-based prediction and the zero-probability problem that motivates smoothing.

N-gram Language Models

Score a held-out token sequence by average negative log-likelihood, then see why perplexity comparisons require the same data, tokenizer, and context window.

Language-Model Evaluation and Perplexity

See how context windows, global co-occurrence counts, and character n-grams shape word vectors, and why nearest-neighbor semantics can mislead.

Word2Vec, GloVe, and Subword Embeddings

How models handle language and speech: count-based language models, smoothing, embeddings, tagging, parsing, translation, speech recognition, and the bridges into neural sequence models.

NLP & Speech

Start here, then follow the prerequisites forward.

Browse the territory.

Notebooks that aren't published yet.