Domain
Representation Learning
How models represent inputs as vectors: similarity and normalization, contrastive training, and sparse autoencoders for reading features out of activations.
Preview the recommended route
Start here. Make one prediction, then note what carries over to the next notebook.
xf_θzgeometryLearnerWhich notebook should I read first?
Open the first notebookRead the first notebook, make one prediction in its demo, then go on to the next.
Recommended route
Start here, then follow the prerequisites forward.
Read these in order: each notebook comes after the notebooks in this domain that it builds on, and the label beside it gives its level. A prerequisite with no page here yet is marked as outside reading. A notebook is left out of the route only when a prerequisite has neither a page here nor a named outside reading; the full list below marks it.
- 01Representation Learning & Embedding Geometry
How models turn inputs into vectors whose geometry can expose useful factors, contextual meaning, and similarity structure.
Intermediate18 mincodedemoafter Scaled Dot-Product Attention & Transformer Layers - 02Sparse Autoencoders: Feature Dictionaries for Mechanistic Interpretability
Sparse autoencoders rewrite each model activation as a sparse combination of learned directions, which are often easier to interpret than individual neurons.
Advanced16 mincodedemoafter Representation Learning & Embedding Geometry, superposition, probingSparse Autoencoders builds directly on Representation Learning & Embedding Geometry.
All notebooks
Every published notebook in this domain.
Representation Learning & Embedding Geometry
How models turn inputs into vectors whose geometry can expose useful factors, contextual meaning, and similarity structure.
Sparse Autoencoders: Feature Dictionaries for Mechanistic Interpretability
Sparse autoencoders rewrite each model activation as a sparse combination of learned directions, which are often easier to interpret than individual neurons.