Continuous Function
Start with Attention to Serving
Learn one route first: how attention math becomes KV-cache memory, long-context pressure, and serving tradeoffs — through concepts, runnable code, and a lab that asks for your prediction before it reveals the answer.
Route Overview
livemetadata-resolved
Current question
How does attention become a serving bottleneck?
Bring a paper, claim, or equation, shore up Efficient Attention, then predict which KV-cache term drives memory before the lab reveals the measurement.
01Attention02Efficient Attention03RoPE04FlashAttention05Long Context06LLM Serving07Decoding
Prediction firstWhich memory symbol should move first?
Mem_KV = B * N_layers * T * H_kv * d_head * 2 * bytesBTH_kvbytes
Paper claims stay separate from the toy demos here and from any future benchmark results.
You leave with something saved to revisit, not just a page you read.
Where You Are
First route: Attention to Serving
A new learner should see one concrete path: map or search a claim, inspect the route graph, open Efficient Attention, predict the KV-cache lever, review the claim boundary, and leave a reproduction or repair note attached to the exact claim, equation, or demo it concerns.
HomeStart with a path this browser can remember.
Start the local trail or open the route graph.
01Homenow02Paper Mappernext action03Knowledge Graphnext action04Searchnext action05Attention to Servingnext action+1 more
Scope labels requiredPreviews and browser-local state are labeled; claims get review before any comparison; reproduction notes stay attached to the exact claim, equation, or demo they describe.
This early version makes no benchmark, hosted-compute, automatic expert review, or live runtime-performance claims.
How every concept is taught
Intuition first, then the math, the code, and a demo you can poke
The site is not an archive of notes. It is a repeatable notebook format for turning abstract ML ideas into something you can reason through from first contact to implementation.
Start with motion, shape, and analogy.
Each concept opens with a mental model you can carry before the notation starts.
Geometry, routing, density, and flow before formalism.
Study Dot ProductWrite the objects down precisely.
Definitions, symbols, and derivations stay close to the intuition instead of replacing it.
Derivatives, KL, vector spaces, and chain rule done step by step.
Study Chain RuleMatch notation to runnable Python.
The symbols on the page become short NumPy or PyTorch fragments you can actually run.
Code mirrors the math instead of hiding it behind frameworks.
Study Adam OptimizerManipulate the system and watch it respond.
Interactive diagrams turn abstract machinery into something you can stress-test, poke, and break.
Attention, diffusion, routing, and serving concepts become explorable.
Study Scaled Dot-Product Attention & Transformer LayersCurated Entries
Start with a thread, not a random page
You can browse the full atlas, but the fastest way in is to follow one editorial thread from prerequisites to modern applications.
track
Foundations First
Build the minimum mathematical language needed to understand optimization and modern model training.
Browse all of Foundations Firsttrack
Transformer Systems
Move from attention mechanics to the engineering decisions that make long-context inference work.
Dot ProductScaled Dot-Product Attention & Transformer LayersEfficient Attention at Scale: KV Cache, GQA & FlashAttentionLong Context Engineering: RoPE Scaling, KV Compression & Memory Optimization
Browse all of Transformer Systemstrack
Frontier Notebooks
Use the foundations to step into generation, alignment, and inspectable model representations.
Scaled Dot-Product Attention & Transformer LayersSparse Autoencoders: Feature Dictionaries for Mechanistic InterpretabilityDirect Preference OptimizationDiffusion, Score-Based Models & Flow Matching
Browse all of Frontier NotebooksDomain Atlas
Navigate by mathematical territory
linear-algebra
Linear Algebra
Vectors, matrices, and linear maps: the language of representations, optimization, and modern deep learning.
5 published5 published demo notebooks11 mapped
Dot ProductMatrix Decompositions: Eigendecomposition, SVD, and Spectral StructureOrthogonality, Projections, and Least-Squares Geometry
calculus
Calculus
Rates of change and accumulation: the language behind gradients, optimization, continuous-time dynamics, and why backprop works as efficiently as it does.
4 published4 published demo notebooks8 mapped
BackpropagationComputation GraphsDerivatives
optimization
Optimization
How we train models: gradients, learning rates, curvature, and the practical tricks that make deep nets converge.
4 published4 published demo notebooks14 mapped
Adam OptimizerGradient DescentLearning Rate Schedules: Warmup, Decay & Cycling
machine-learning
Machine Learning
The classical supervised-learning spine: models, losses, generalization, evaluation, and the experiment habits that make modern AI results trustworthy.
10 published10 published demo notebooks78 mapped
Bias-Variance DecompositionClassification Metrics, Thresholds, and CalibrationLinear Regression & Least Squares
probability
Probability
Uncertainty made precise: events, random variables, expectations, and the distributions that models learn.
6 published6 published demo notebooks15 mapped
Cross-EntropyDistributionsMaximum Likelihood
information-theory
Information Theory
How we measure information and mismatch between distributions: entropy, cross-entropy, KL divergence, mutual information, and why they appear everywhere in ML.
1 published1 published demo notebooks2 mapped
KL Divergence (Relative Entropy)
attention-transformers
Attention & Transformers
The sequence model backbone: tokenization, self-attention, positional encodings, and the transformer block that powers modern LLMs.
9 published9 published demo notebooks20 mapped
Scaled Dot-Product Attention & Transformer LayersEfficient Attention at Scale: KV Cache, GQA & FlashAttentionFlashAttention: IO-Aware Attention
nlp-speech
NLP & Speech
How models handle language and speech: count-based language models, smoothing, embeddings, tagging, parsing, translation, speech recognition, and the bridges into neural sequence models.
0 published0 published demo notebooks5 mapped
notebooks in review
representation-learning
Representation Learning
Embeddings and the geometry of meaning: similarity, normalization, contrastive objectives, and why vector spaces become usable interfaces for models.
3 published3 published demo notebooks3 mapped
Autoencoders and Denoising AutoencodersRepresentation Learning & Embedding GeometrySparse Autoencoders: Feature Dictionaries for Mechanistic Interpretability
generative-models
Generative Models
How models generate: likelihood, latent variables, diffusion/score models, flows, and the training tricks that make sampling work.
5 published5 published demo notebooks6 mapped
Diffusion, Score-Based Models & Flow MatchingFlow Matching & Rectified FlowsNormalizing Flows: Tractable Density via Invertible Transforms
scaling
Scaling
How loss and capability change with parameters, data, and compute; how to allocate a training budget; and why some abilities appear suddenly at scale.
2 published2 published demo notebooks6 mapped
Overparameterization & Generalization (Double Descent)Scaling Laws & Emergent Abilities
reinforcement-learning
Reinforcement Learning
How agents choose actions over time: states, actions, rewards, transition dynamics, value functions, exploration, and the policy-optimization ideas behind modern RL and preference learning.
0 published0 published demo notebooks13 mapped
notebooks in review
alignment
Alignment
How we shape model behavior: preference learning, reward modeling, KL-regularized fine-tuning, and the failure modes that appear when you optimize the wrong thing.
5 published5 published demo notebooks5 mapped
Direct Preference OptimizationKahneman-Tversky OptimizationProcess Reward Models: Step-Level Verifiers for Reasoning
efficiency
Efficiency
How we make models cheaper to train and serve: quantization, distillation, low-rank adapters, sparsity, and the memory/latency tradeoffs that dominate real deployments.
5 published5 published demo notebooks5 mapped
Efficiency: Quantization, Distillation, LoRA & Sparse MoEKnowledge Distillation: Learning from TeachersPruning: Removing Unnecessary Weights
llm-systems
LLM Systems
How models run in production: prefill vs decode, KV cache memory, batching and scheduling, and the techniques that make latency and throughput practical.
5 published5 published demo notebooks10 mapped
Decoding & Sampling: Temperature, Top-p & Inference-Time ControlLLM Serving at Scale: Prefill, Decode & Continuous BatchingMoE Serving & Scheduling: Token Dispatch, All-to-All, Disaggregated Parallelism
production-ml
Production ML
How we keep deployed models trustworthy: evaluation pipelines, dataset and model versioning, drift monitoring, reproducibility, and the operational tradeoffs between them.
4 published4 published demo notebooks7 mapped
Cost and Latency ObservabilityEvaluation Harnesses and Benchmark ContaminationEvaluation Pipelines
Paper Mapper
Turn a paper clue into a route, one equation, and one experiment.
Clickable equation
Mem_KV = B * N_layers * T * H_kv * d_head * 2 * bytesThe mapper turns the bottleneck into a calculator, then links the symbols back to attention and serving.
Example route note
What changed from prior work?
The paper trades exact key-value retention for a bounded decode-time memory budget. Learn attention first, test the memory curve, then inspect where retrieval quality can break.
Extract equationsFind prerequisitesOpen a labAsk one question
Learner, Researcher, Professor
One atlas, different depths of use.
A guided route through prerequisites, notation, code, and demos.
Follow a path, predict what each demo will do, then ask the AI companion about the step you got wrong.
Start with linear algebraA quick way to inspect the mechanism, its assumptions, and runnable code.
Use concept pages as small, testable models before jumping back to papers or experiments.
Inspect process reward modelsA teachable sequence with visual hooks, derivation checkpoints, and failure regimes.
Teach from the page itself: intuition first, then math, code, and a demo students can drive.
Open maximum likelihoodFrom Claim to Experiment
Map AI research to a route you can test.
Move from a claim or equation to the prerequisites it leans on, a small runnable demo, an honest note on what that demo shows, and one sharper question — without losing the thread.
01Claim
Map the claim
Paste a specific equation, architecture name, arXiv ID, or short excerpt to find the prerequisite path in the atlas.
02Map
Reveal prerequisites
The graph places the paper inside concepts, equations, prior work, and the ideas you should repair first.
03Lab
Run a toy demo
Small authored examples turn the central mechanism into sliders, curves, tensors, and small failure cases.
04Discuss
Ask one sharper question
Discussion prompts attach to the exact concept, equation, lab, or paper claim so the next conversation has a clear anchor.
The Guided Route
Attention to Serving, start to finish.
One complete path through transformer inference — from the attention equation to the tradeoffs of serving it — for students, engineers, and researchers.
Open Attention to ServingAsk Beside the Notebook
Specific help attached to the route.
Concept Coach
Breaks down notation, prerequisites, equations, demos, and the current learner question.
Paper Mapper
Maps a pasted claim, equation, arXiv ID, or model-report excerpt to concepts, sources, and caveats.
Demo Guide
Points to authored browser demos and small local examples that test one mechanism at a time.
Claim Scope Review
Separates what a toy demo shows from what depends on sources, scale, or expert review.
AI Companion
Guided lessons, rigorous notes, coding practice, and a companion attached to the object you are studying.
A mathematical playground, a serious course, a code lab, and a patient tutor — in the same place. The companion helps you ask sharper questions, recover missing prerequisites, turn notation into code, and test understanding against the exact equation or live demo in view.
AI Companion
Plan beside the atlas
Choose a path, study a notebook, play with the demo — then ask the companion to explain, quiz, connect, or debug whatever you have selected.
Context prompt
You are my AI learning companion for Continuous Function. Current context: homepage learning atlas. Learning surface: Continuous Function atlas. What this page says: Choose a path, study a notebook, play with the demo — then ask the companion to explain, quiz, connect, or debug whatever you have selected. Suggested next step: Pick a track and ask the companion to turn it into a short plan.. Learner goal: Understand the idea. Learner comfort level: New to this. Preferred explanation style: Visual first. Task: Ask me 5 quick questions, then recommend the best Continuous Function learning path for my current level. Answer in a way that helps me learn: ask one clarifying question only if needed, use intuition before notation, and end with one thing I should try on the page.