AI Lab Curriculum

A connected route from foundations to frontier work

The first launch spine is not a link directory. It is a local curriculum that studies the best open books, courses, notebooks, codebases, and research threads, then turns their strongest ideas into original object-grounded routes, labs, and next moves.

6 launch tracks26 source families28 academic anchorslocal-first contentdeep links only

Run Diagnostic Open Tracks Academic Spine Start Transformer Lab Inspect Sources

Local curriculum spineQuestion -> Object -> Witness -> Invariant

01Math object

vector, loss, gradient

02Model mechanism

attention, score, circuit

03Lab witness

predict, run, inspect

04Research object

paper claim, source span

Selected objectQK^T / sqrt(d_k) -> softmax -> V

One equation becomes prerequisites, code, an experiment, a paper route, and a next question.

Local diagnostic

Start with the object you are trying to understand.

This is not an account gate or a placement quiz. It turns your current confusion into a local route, then keeps the selected object, check, evidence, invariant, and next move together.

What are you trying to understand?

Where is the work blocked?

Use boundary

Local recommendation only. It uses the curated AI Lab routes already on this page; no login or data service is involved.

Sample route

Transformer Lab

Open route

Selected objectA token route through embeddings, attention scores, value aggregation, KV memory, and decoded outputs.

Current questionStart from a paper, equation, model behavior, architecture, or system tradeoff that feels opaque.

Boundary

Use the local route first; source checks, graph, and memory previews stay available as follow-up surfaces.

Check

Predict how KV memory changes when context length, head grouping, or precision changes.

Evidence

Show token lanes, attention weights, cache size, latency proxy, and sampled outputs side by side.

Invariant

Attention quality, memory movement, and decoding behavior are coupled through exact tensor objects.

First object to inspectToken, embedding, positional signal

What information is present before attention starts?

Route memoryChecking local route

Reading browser memory before deciding whether this route can be saved.

No diagnostic signal yet; this flagship route shows how a complete local loop feels.

All routes remain available

Math CoreWhich mathematical objects keep reappearing when a modern model seems mysterious?DL CoreHow do data, loss, gradients, representations, and optimization become a trainable network?GenerativeHow do modern generative models define, corrupt, transport, or reverse probability mass?AlignmentHow do feedback, reward, preference, features, and circuits shape model behavior?SystemsHow do serious AI projects turn ideas into reproducible experiments, evaluations, and deployable systems?

Start this route Inspect route card Search this object

Launch promise

Make modern AI ideas operational, not merely familiar.

A serious learner should be able to arrive with a paper, equation, architecture, training behavior, or system tradeoff and leave with a route: prerequisite repair, exact object, prediction, manipulation, evidence, invariant, and next experiment.

01Question

02Object

03Prediction

04Manipulation

05Evidence

06Invariant

07Next move

Flagship local route

Transformer Lab: start from text to one update.

Put one tiny decoder case on the bench, predict a gradient path, inspect the exact evidence, then carry the invariant into RoPE, KV memory, long context, and serving.

Checking local routeWhich source token receives the largest gradient signal from the final next-token loss?

1Selected object
2Prereq repair
3Predict
4Manipulate
5Evidence
6Invariant
7Next

Selected lab objectFrom text to one gradient updatetext -> toy tokens -> causal attention -> softmax CE -> dL/dz

Before opening evidence, predict which source token receives the largest gradient signal.

Objectthe model learns patterns

Manipulatetokenizer, key scale, temperature, mask, residual

Invariant to carryGradient follows actual forward dependency paths.

Prediction instrumentCommit first, then inspect the forward and backward trace.

changedToy tokenizercoarse -> fine

changedmodel key scale1x -> 4x

held fixedResidual pathon first

EvidenceAttention scores, probabilities, value paths, residual path, loss, and finite difference.

01Text to one updateToy tokens, causal route, loss, gradient 02Tokens and positionToken ids, embeddings, RoPE phase 03Attention routingQK^T scores, mask, softmax row 04RoPE phaseRotated Q/K pair, phase gap, dot score 05KV memoryMem_KV = B * N_layers * T * H_kv * d_head * 2 * bytes 06Long-context pressureMemory, position, retrieval, eval limits 07Serving and decodingTTFT, TPOT, cache reads, sample support 08Speculative decodingDraft-target verification and accepted prefix 09Evaluation and falsificationClaim, slice, metric, counterexample 10Capstone systems claimAssumptions, evidence, failure boundary

Curriculum tracks

Start with six routes that can become excellent before the platform gets more complex.

Math CoreOpen route

Mathematical Core for Modern AI

Which mathematical objects keep reappearing when a modern model seems mysterious?

The learner can recognize vectors, derivatives, probabilities, losses, and optimization steps inside model behavior.

Central objectA model computation as a chain of vector spaces, maps, losses, and update rules.

Vector Spaces Dot Product Derivatives Probability Basics Gradient Descent

Represent the stateVector, basis, dot product, norm

What changes when I switch coordinate systems but keep the same object?

Measure changeDerivative, chain rule, computation graph

Where does the gradient signal actually flow?

Turn fit into probabilityLikelihood, cross-entropy, uncertainty

What does the loss assume about the data and the model?

First lab shape

Predict: Predict which variable changes the loss curvature before plotting it.

Manipulate: Move a point, basis, probability, or step size while holding the invariant fixed.

Evidence: Show the same object in words, equation, NumPy/PyTorch code, and a small chart.

Invariant: A model update is a local measurement plus a rule for moving through parameter space.

DL CoreOpen route

ML and Deep Learning Core

How do data, loss, gradients, representations, and optimization become a trainable network?

The learner can implement and debug a small network while explaining each signal in the math.

Central objectA trainable computation graph with a loss, gradient, optimizer state, and representation.

Backpropagation Reverse-Mode Autodiff Gradient Descent Adam Representations

Build the graphForward computation and cached intermediates

Which local values must be remembered for the backward pass?

Move the weightsGradient descent, momentum, Adam, weight decay

Which optimizer state changes the path without changing the objective?

Inspect representationHidden features as useful coordinates

What does the network make linearly available after training?

First lab shape

Predict: Predict which change makes training unstable: step size, normalization, initialization, or data scale.

Manipulate: Toggle optimizer and normalization choices on a tiny network.

Evidence: Compare loss curves, gradient norms, and representation snapshots.

Invariant: Training works when useful error signals survive the graph and update parameters at a scale the model can absorb.

Transformer LabOpen route

Transformer and Language Model Lab

How does a language model turn tokens into context-aware predictions under memory and compute constraints?

The learner can move from tokenization to attention, RoPE, KV cache, decoding, and serving tradeoffs.

Central objectA token route through embeddings, attention scores, value aggregation, KV memory, and decoded outputs.

Attention and Transformers Positional Encoding RoPE Efficient Attention LLM Serving

Token to representationToken, embedding, positional signal

What information is present before attention starts?

Context routingQK^T scores, softmax weights, value mixing

Which token is allowed to influence this position, and by how much?

Inference pressureKV cache, decoding distribution, latency

Which symbol changes memory first when context length grows?

First lab shape

Predict: Predict how KV memory changes when context length, head grouping, or precision changes.

Manipulate: Change sequence length, group size, decoding settings, and serving batch assumptions.

Evidence: Show token lanes, attention weights, cache size, latency proxy, and sampled outputs side by side.

Invariant: Attention quality, memory movement, and decoding behavior are coupled through exact tensor objects.

GenerativeOpen route

Generative Model Mechanisms

How do modern generative models define, corrupt, transport, or reverse probability mass?

The learner can compare VAEs, flows, diffusion, score matching, and flow matching by mechanism.

Central objectA data distribution transformed through latent variables, noise schedules, scores, or vector fields.

Diffusion Score Matching Flow Matching Maximum Likelihood

Choose the probability objectDensity, latent variable, score, or transport field

What is the model learning to estimate?

Manipulate the pathNoise schedule or vector field

Which direction restores structure from noise?

Compare familiesLikelihood, sample path, training target, inference cost

What does each generative family make easy or hard?

First lab shape

Predict: Predict whether a point should move toward high-density data or away from noise.

Manipulate: Slide time, noise, vector field strength, and sampling steps.

Evidence: Show sample path, score/vector arrows, objective target, and failure regime.

Invariant: A generative model is a learned rule for moving probability mass or reconstructing structure.

AlignmentOpen route

Alignment and Interpretability Workbench

How do feedback, reward, preference, features, and circuits shape model behavior?

The learner can separate training signal, behavioral claim, internal feature, and source evidence.

Central objectA model behavior explained through reward signal, preference pair, feature, circuit, or failure mode.

RLHF Direct Preference Optimization Reward Hacking Sparse Autoencoders Representations

Shape behaviorReward, preference, policy update

What signal is actually being optimized?

Inspect internalsFeature, activation, attention head, circuit

What evidence links this internal object to the behavior?

Bound the claimSource span, demo witness, failure case

What is known, inferred, or only locally demonstrated?

First lab shape

Predict: Predict whether a feedback signal improves behavior or opens a reward-hacking loophole.

Manipulate: Change reward definition, preference margin, feature sparsity, or verification rule.

Evidence: Compare behavior, objective value, feature activation, and caveat state.

Invariant: Alignment claims need both external behavior evidence and exact training/internal objects.

SystemsOpen route

Systems and Research Practice

How do serious AI projects turn ideas into reproducible experiments, evaluations, and deployable systems?

The learner can plan an experiment, read a paper into objects, run a minimal witness, and name the next risk.

Central objectA research claim turned into source span, equation, code witness, measurement, and next experiment.

Pretraining Data Mixtures LLM Serving Retrieval-Augmented Generation Structured Decoding Quantization

Map the claimPaper span, equation, benchmark, assumption

Which part of the paper should become a local test?

Run the witnessSmall reproducible code path

What is the smallest experiment that can falsify my reading?

Ship with caveatsLatency, monitoring, evaluation, cost, safety boundary

Which engineering constraint changes the model choice?

First lab shape

Predict: Predict which constraint breaks first: quality, cost, latency, data drift, or evaluation reliability.

Manipulate: Change batch, context, retrieval, quantization, and evaluation thresholds.

Evidence: Show metric movement, caveat, and a reproducible next experiment.

Invariant: Research understanding compounds when every claim becomes an object, a witness, and a next test.

Academic spine

Keep the depth of papers and monographs, then translate each one into an object a learner can work with.

This layer is the launch canon for rigorous source grounding. It brings textbooks, theory, canonical papers, and research programs into one route system while preserving the learning loop: intuition, math, code, demo, evidence, and invariant.

Theory textOpen source

Statistical learning foundations

The Elements of Statistical Learning

Depth object: Bias-variance, regularization paths, kernels, trees, boosting, and unsupervised structure.
Why it matters: It is a broad, rigorous bridge from classical statistical learning to the model families that deep learning builds on or absorbs.
Intuitive bridge: Turn each method family into a small decision surface: what object is fit, what is regularized, and what failure mode appears.
Use boundary: Use the official author page as the link target; cite and synthesize rather than copying text or figures.

Maximum Likelihood Cross-Entropy Representations Loss Landscapes

MonographOpen source

Optimization theory

Convex Optimization

Depth object: Convex sets, duality, constrained optimization, and reliable numerical procedures.
Why it matters: It gives a precise language for objectives, constraints, dual variables, and what makes an optimization problem tractable.
Intuitive bridge: Teach convexity as a geometry of no hidden traps before contrasting it with deep-network loss landscapes.
Use boundary: The official site notes Cambridge allowed the book to stay available on the web; still link/cite instead of mirroring.

Gradient Descent Loss Landscapes Weight Decay and AdamW Adam

MonographOpen source

Information and inference

Information Theory, Inference, and Learning Algorithms

Depth object: Entropy, coding, Bayesian inference, graphical models, neural networks, and error-correcting codes.
Why it matters: It unifies probability, information, and learning in a way that makes modern objectives less arbitrary.
Intuitive bridge: Use small coding and uncertainty games before formal KL, entropy, likelihood, and posterior objects.
Use boundary: Link to the official book PDF; retain copyright boundaries and write original explanations.

Probability Basics Bayesian Inference Cross-Entropy kl divergence

MonographOpen source

Bayesian nonparametrics

Gaussian Processes for Machine Learning

Depth object: Distributions over functions, kernels, marginal likelihood, posterior prediction, and uncertainty.
Why it matters: It makes function-space thinking concrete, which helps with kernels, NTK, Bayesian inference, and model uncertainty.
Intuitive bridge: Show a learner how changing a kernel changes a distribution over functions before presenting the algebra.
Use boundary: The site says MIT Press allowed the web version; deep link and cite, do not redistribute chapters.

Bayesian Inference Neural Tangent Kernel Representations Probability Basics

Theory textOpen source

Learning theory

Foundations of Machine Learning

Depth object: Generalization bounds, Rademacher complexity, kernels, online learning, and boosting.
Why it matters: It gives serious learners the proof objects behind claims about generalization and learnability.
Intuitive bridge: Introduce bounds as stress tests on what a training result can legitimately imply.
Use boundary: Use the official author page and MIT Press online versions; avoid reproducing text or exercises.

Scaling Laws Loss Landscapes Gradient Descent Representations

Theory textOpen source

Algorithmic learning theory

Understanding Machine Learning: From Theory to Algorithms

Depth object: PAC learning, uniform convergence, convex learning, regularization, and online learning.
Why it matters: It explains why learning can be possible at all, not just how to run a training loop.
Intuitive bridge: Frame each theorem as a claim about what evidence training data can or cannot provide.
Use boundary: Prefer official author/course links; cite the book and write local concept-specific bridges.

Gradient Descent Loss Landscapes Maximum Likelihood Representations

MonographOpen source

Reinforcement learning

Reinforcement Learning: An Introduction

Depth object: Value functions, policy improvement, temporal-difference learning, planning, and function approximation.
Why it matters: It is the conceptual backbone for feedback, reward, exploration, and later RLHF-related ideas.
Intuitive bridge: Start with a tiny choice process and show how value estimates become behavior-shaping objects.
Use boundary: Link to the official book page; avoid copying chapters, figures, or exercises into local pages.

RLHF Reward Hacking Process Reward Models Tree Search Reasoning

Theory textOpen source

Probabilistic ML

Pattern Recognition and Machine Learning

Depth object: Bayesian decision theory, graphical models, kernels, mixture models, variational inference, and EM.
Why it matters: It is one of the clearest academic bridges from probability to pattern-recognition algorithms.
Intuitive bridge: Use it as a map of inference families, then attach each family to one small code witness and one diagram.
Use boundary: Use official Springer or author-hosted links where available; do not mirror PDFs from unofficial repositories.

Bayesian Inference Maximum Likelihood Normalizing Flows Variational Autoencoders

Canonical paperOpen source

Transformer foundations

Attention Is All You Need

Depth object: Scaled dot-product attention, multi-head attention, positional encoding, and sequence-to-sequence architecture.
Why it matters: It is the root paper for the architecture family behind modern LLMs.
Intuitive bridge: Make Q, K, V, score matrix, softmax, and value mixing separate manipulable objects before reading the paper.
Use boundary: Link to NeurIPS/arXiv pages and cite; use local diagrams and original code witnesses.

Attention and Transformers Dot Product Positional Encoding Residual Connections

Canonical paperOpen source

Efficient attention systems

FlashAttention

Depth object: IO-aware exact attention, tiling, SRAM/HBM movement, and long-sequence scaling.
Why it matters: It teaches that asymptotic FLOPs are not the whole story; memory movement can dominate real systems.
Intuitive bridge: Use a cache-movement lab where learners predict what changes when sequence length and tile size change.
Use boundary: Link to NeurIPS/OpenReview/arXiv; do not copy figures or code unless license is checked.

FlashAttention Efficient Attention Long Context LLM Serving

Canonical paperOpen source

Positional structure

RoFormer: Enhanced Transformer with Rotary Position Embedding

Depth object: Rotations that encode absolute position while making relative position visible in attention.
Why it matters: RoPE is now a standard positional mechanism in language models and long-context extensions.
Intuitive bridge: Teach RoPE as rotating query/key vectors on a plane, then show how dot products depend on relative offset.
Use boundary: Link to arXiv and local concept pages; write original derivations and visual witnesses.

RoPE Positional Encoding Attention and Transformers Long Context

Canonical paperOpen source

Scaling behavior

Scaling Laws for Neural Language Models

Depth object: Power-law relations between loss, model size, dataset size, and compute.
Why it matters: It turns scale from a vague intuition into measurable empirical laws.
Intuitive bridge: Use log-log plots where learners predict which curve bends when compute, data, or parameters change.
Use boundary: Link to arXiv; keep local explanations source-bounded and updated when later scaling work changes the picture.

Scaling Laws Pretraining Data Mixtures Cross-Entropy Loss Landscapes

Canonical paperOpen source

Compute-optimal training

Training Compute-Optimal Large Language Models

Depth object: Tradeoff between parameters, tokens, compute budget, and downstream performance.
Why it matters: It corrected a widely held scaling intuition: many large models were undertrained on tokens.
Intuitive bridge: Make a budget allocation simulator before asking learners to parse the fitted laws.
Use boundary: Link to arXiv/NeurIPS; explain as one point in a living scaling-law thread, not final scripture.

Scaling Laws Pretraining Data Mixtures LLM Serving efficiency

Canonical paperOpen source

Optimization algorithms

Adam: A Method for Stochastic Optimization

Depth object: Adaptive first and second moment estimates for stochastic gradients.
Why it matters: Adam is a default optimizer in modern deep learning, but its behavior is often treated as magic.
Intuitive bridge: Show moving averages as two memory traces that change the update direction and scale.
Use boundary: Link to arXiv/ICLR; keep implementation snippets minimal and notation-aligned.

Adam Gradient Descent learning rate schedules Weight Decay and AdamW

Canonical paperOpen source

Training stabilization

Batch Normalization

Depth object: Mini-batch mean/variance normalization inside a network layer.
Why it matters: It changed practical training by stabilizing activations and enabling higher learning rates.
Intuitive bridge: Use a layer-distribution witness before and after normalization, then discuss what the original explanation missed.
Use boundary: Link to PMLR and cite; keep local caveats honest because later work revises the internal-covariate-shift story.

Batch Normalization Gradient Descent Backpropagation Representations

Canonical paperOpen source

Deep architecture optimization

Deep Residual Learning for Image Recognition

Depth object: Residual functions, skip connections, optimization degradation, and depth scaling.
Why it matters: Residual connections are now a basic architectural primitive far beyond vision models.
Intuitive bridge: Make the skip path a visible information highway and ask what the residual branch must learn.
Use boundary: Use CVF open-access link; create original diagrams and explanations.

Residual Connections Backpropagation Representations Attention and Transformers

Canonical paperOpen source

Training dynamics theory

Neural Tangent Kernel

Depth object: Function-space gradient descent in the infinite-width limit.
Why it matters: It gives a mathematically precise lens on why very wide networks can train like kernel methods.
Intuitive bridge: Show a network function moving in output space before introducing the limiting kernel formalism.
Use boundary: Link to NeurIPS/arXiv; keep the scope carefully bounded to the assumptions of the theorem.

Neural Tangent Kernel Gradient Descent Representations Scaling Laws

Canonical paperOpen source

Latent-variable generative models

Auto-Encoding Variational Bayes

Depth object: Variational lower bound, recognition model, reparameterization trick, and amortized inference.
Why it matters: It made neural latent-variable models trainable at scale.
Intuitive bridge: Start with the impossible posterior, then show the approximate posterior as a learned shortcut.
Use boundary: Link to arXiv/ICLR record; local derivations should define every distribution and expectation.

Variational Autoencoders Bayesian Inference kl divergence Maximum Likelihood

Canonical paperOpen source

Diffusion models

Denoising Diffusion Probabilistic Models

Depth object: Forward noising process, learned reverse denoising process, variational objective, and sampling chain.
Why it matters: It is a core reference for the diffusion-model family behind modern image and multimodal generation.
Intuitive bridge: Let learners corrupt a point cloud step by step, then predict the reverse score/denoising direction.
Use boundary: Link to NeurIPS; use original local visuals and toy distributions.

Diffusion Score Matching Probability Basics Flow Matching

Canonical paperOpen source

Continuous-time generative modeling

Score-Based Generative Modeling through Stochastic Differential Equations

Depth object: Forward SDE, reverse-time SDE, score field, and numerical sampling.
Why it matters: It unifies score-based and diffusion-style generative modeling in a continuous-time mathematical language.
Intuitive bridge: Teach the score as an arrow field on noisy data before formal SDE notation.
Use boundary: Link to OpenReview/arXiv; keep stochastic-process assumptions explicit.

Score Matching Diffusion Flow Matching Probability Basics

Canonical paperOpen source

Flow-based generative modeling

Flow Matching for Generative Modeling

Depth object: Probability paths, conditional vector fields, continuous normalizing flows, and simulation-free training.
Why it matters: It gives a clean alternative view of generative modeling as learning velocity fields between distributions.
Intuitive bridge: Make the model a field of arrows that transports simple noise into data before discussing CNF objectives.
Use boundary: Link to OpenReview; use local low-dimensional transport demos for explanation.

Flow Matching Normalizing Flows Diffusion Score Matching

Canonical paperOpen source

Alignment and feedback

Training Language Models to Follow Instructions with Human Feedback

Depth object: Supervised fine-tuning, reward modeling, PPO-style RLHF, and human preference data.
Why it matters: It made human-feedback alignment a central practical workflow for language models.
Intuitive bridge: Separate the three data objects: demonstrations, rankings, and policy updates.
Use boundary: Link to arXiv/OpenAI material; keep claims scoped to the paper and later methods.

RLHF Reward Hacking Process Reward Models Direct Preference Optimization

Canonical paperOpen source

Preference optimization

Direct Preference Optimization

Depth object: Preference-pair classification objective derived from reward-policy duality.
Why it matters: It gives a simpler alignment training route than full RLHF for many settings.
Intuitive bridge: Show one chosen/rejected pair and how the log-ratio changes before deriving the objective.
Use boundary: Link to NeurIPS/OpenReview/arXiv; local math must preserve assumptions behind the derivation.

Direct Preference Optimization RLHF Cross-Entropy Reward Hacking

Canonical paperOpen source

Reasoning supervision

Let's Verify Step by Step

Depth object: Step-level reward models, process supervision, active learning, and verification of reasoning traces.
Why it matters: It clarifies the difference between rewarding final answers and rewarding the reasoning process.
Intuitive bridge: Use a tiny multi-step proof where learners predict which step should receive feedback.
Use boundary: Link to OpenAI-hosted PDF; summarize source-bounded claims and avoid copying figures.

Process Reward Models Tree Search Reasoning RLHF Reward Hacking

Canonical paperOpen source

Mechanistic interpretability

Sparse Autoencoders Find Highly Interpretable Features in Language Models

Depth object: Sparse feature dictionaries over internal activations and causal feature attribution.
Why it matters: It makes feature discovery a scalable, inspectable route into model internals.
Intuitive bridge: Begin with overcomplete dictionary learning on toy vectors before moving to transformer activations.
Use boundary: Link to arXiv/OpenReview; use local toy models before discussing real-model claims.

Sparse Autoencoders Representations Attention and Transformers Scaling Laws

Research programOpen source

Mechanistic interpretability

A Mathematical Framework for Transformer Circuits

Depth object: Residual stream, attention heads, virtual weights, paths, and circuit-level decomposition.
Why it matters: It gives one of the clearest mathematical languages for reverse-engineering transformers.
Intuitive bridge: Make the residual stream a physical shared workspace where heads write and read information.
Use boundary: Link to the Transformer Circuits thread; cite and create original diagrams/labs.

Attention and Transformers Representations Sparse Autoencoders Residual Connections

Research programOpen source

In-context learning mechanisms

In-context Learning and Induction Heads

Depth object: Attention-head circuits that copy or complete repeated token patterns.
Why it matters: It connects a visible behavioral capability, in-context learning, to an interpretable transformer circuit.
Intuitive bridge: Use a repeated-token toy sequence and let learners predict where the second attention head should look.
Use boundary: Link to the official article/arXiv; keep local claims separated into causal, correlational, and toy evidence.

Attention and Transformers Tokenization and Vocabulary Representations Sparse Autoencoders

Research programOpen source

Representation geometry

Toy Models of Superposition

Depth object: Sparse features represented in fewer dimensions through superposition and interference.
Why it matters: It gives a geometric explanation for why individual neurons can be polysemantic.
Intuitive bridge: Show two features competing for one dimension before introducing phase changes and polytopes.
Use boundary: Link to official thread/arXiv; build local toy demos rather than copying figures.

Sparse Autoencoders Representations Vector Spaces Dot Product

Source map

Study the best materials for their teaching moves, then synthesize original routes.

The stance is strict: link, cite, and learn from structure. Do not mirror books, PDFs, slides, figures, assignments, or code unless the license explicitly allows it.

Book Course Notebook Codebase Engineering guide Visual essay Paper thread

BookVisit source

Dive into Deep Learning

Interactive book

Deep learning from math to code across PyTorch, JAX, TensorFlow, and NumPy-style implementations.

Why study it: It shows how an open book can keep formulas, runnable code, exercises, and discussion in one tight loop.
Borrow in spirit: Mirror the code-math adjacency, but add stronger selected-object routing and prediction-first labs.
Use policy: Deep link, cite, and synthesize. Do not copy chapters, figures, or code blocks into canonical pages.

Gradient Descent Backpropagation Attention and Transformers Diffusion

BookVisit source

Deep Learning

Reference textbook

Mathematical and conceptual background for deep learning, optimization, probability, and architectures.

Why study it: It remains a stable reference for foundational definitions and the shape of the field.
Borrow in spirit: Use it as a completeness checklist, then rewrite locally as shorter object-level routes.
Use policy: Link to the official online book and cite specific chapters; do not redistribute derived excerpts.

Probability Basics Maximum Likelihood Gradient Descent Representations

BookVisit source

Mathematics for Machine Learning

Math bridge

Linear algebra, analytic geometry, matrix decompositions, vector calculus, probability, and optimization.

Why study it: It is one of the clearest bridges from undergraduate math objects to machine-learning objects.
Borrow in spirit: Turn each prerequisite into a repair card attached to the AI object that needs it.
Use policy: Link to the official book site; respect personal-use PDF language and avoid redistribution.

Vector Spaces Dot Product Derivatives Gradient Descent

BookVisit source

Understanding Deep Learning

Modern deep-learning text

A pragmatic treatment of deep learning concepts, modern architectures, and training behavior.

Why study it: It has a fresh scope for modern DL without becoming only a historical survey.
Borrow in spirit: Use its modern topic selection, then make each mechanism manipulable in the browser.
Use policy: Deep link to official material and cite; keep Continuous Function explanations original.

Backpropagation Adam Attention and Transformers Diffusion

BookVisit source

Probabilistic Machine Learning

Probabilistic modeling reference

Modern probabilistic ML, Bayesian modeling, uncertainty, graphical models, and deep generative links.

Why study it: It gives the platform a rigorous spine for uncertainty instead of treating probability as a prereq footnote.
Borrow in spirit: Attach uncertainty and inference assumptions directly to claims, demos, and model behavior.
Use policy: Link to official resources; do not mirror book text or figures unless license permits.

Probability Basics Bayesian Inference Maximum Likelihood Score Matching

CourseVisit source

MIT OCW 18.06 Linear Algebra

Open courseware

Matrix theory, vector spaces, eigenvalues, positive definite matrices, and useful applications.

Why study it: It is the canonical repair path when vector-space intuition blocks deep-learning progress.
Borrow in spirit: Use lecture-level geometric intuition, then connect each object to tensors, attention, and representations.
Use policy: Use official OCW links and license-aware attribution; avoid copying lecture assets wholesale.

Vector Spaces Dot Product Linear Transformations Representations

CourseVisit source

Stanford CS229 Machine Learning

ML foundations course

Classical ML, supervised learning, generalization, optimization, kernels, and probabilistic models.

Why study it: It is the right calibration for rigorous ML fundamentals before frontier architectures.
Borrow in spirit: Convert long lecture arcs into compact "why this object matters now" prerequisite repairs.
Use policy: Link and cite official course pages or notes; keep local derivations original.

Maximum Likelihood Cross-Entropy Gradient Descent Bayesian Inference

CourseVisit source

Stanford CS231n Deep Learning for Computer Vision

Deep-learning systems course

Training neural networks, optimization, backpropagation, CNNs, debugging, and visual recognition.

Why study it: It teaches the mechanics of training and debugging rather than only final architectures.
Borrow in spirit: Borrow the assignment mindset: every mechanism should be implemented and stress-tested.
Use policy: Link to official notes and materials; do not copy assignments or solutions into the site.

Backpropagation Gradient Descent Adam Representations

CourseVisit source

Stanford CS224N NLP with Deep Learning

NLP and LLM course

Word vectors, sequence models, attention, transformers, and modern LLM topics.

Why study it: It is a strong bridge from foundational DL to language-model mechanisms.
Borrow in spirit: Use it to choose route order, while making Continuous Function more object-attached and interactive.
Use policy: Deep link and cite official lectures/readings; do not reproduce course assets without permission.

Attention and Transformers Positional Encoding RoPE Decoding and Sampling

CourseVisit source

Stanford CS336 Language Modeling from Scratch

Frontier language-model course

The process of building language models from tokenizer and data to training, scaling, and evaluation.

Why study it: It is unusually close to how serious modern AI labs think about full-stack model building.
Borrow in spirit: Make "from scratch" a route spine: object, code witness, measurement, scaling constraint, paper tie-in.
Use policy: Link to current and archived official offerings; do not mirror assignments or course files.

Tokenization and Vocabulary Attention and Transformers Scaling Laws Pretraining Data Mixtures

CourseVisit source

fast.ai Practical Deep Learning for Coders

Top-down practical course

Practical model building with fast feedback and runnable notebooks.

Why study it: It shows the motivational power of building something real before all theory is known.
Borrow in spirit: Use top-down momentum, then add precise repair docks for the math that appears under the hood.
Use policy: Link to the course and notebooks; do not copy lesson text or notebooks into canonical content.

Representations Backpropagation Gradient Descent Diffusion

NotebookVisit source

UvA Deep Learning Tutorials

Notebook course

PyTorch-oriented tutorial notebooks for optimization, transformers, graph neural networks, and more.

Why study it: It demonstrates how notebooks can teach theory through executable implementation detail.
Borrow in spirit: Borrow the notebook-as-lab rhythm, but expose invariants and concept routes in the website UI.
Use policy: Deep link to notebooks and docs; cite; keep local examples small and original.

Gradient Descent Attention and Transformers Representations Sparse Autoencoders

CourseVisit source

Hugging Face Learn

Open ecosystem curriculum

LLMs, transformers, agents, diffusion, deep RL, computer vision, and practical libraries.

Why study it: It is the practical ecosystem layer learners need after understanding the mechanism.
Borrow in spirit: Use ecosystem breadth as route destinations, not as a substitute for mechanism-first learning.
Use policy: Link to course chapters and docs; avoid copying exercises, datasets, or model cards.

Attention and Transformers Decoding and Sampling Diffusion RLHF

NotebookVisit source

The Annotated Transformer

Annotated implementation

Line-by-line implementation of the Transformer paper as a working notebook.

Why study it: It is a classic example of turning a paper into executable understanding.
Borrow in spirit: Turn each paper mechanism into aligned equation, code witness, demo state, and source claim.
Use policy: Link to the post/repo; use only license-compatible snippets with attribution if ever needed.

Attention and Transformers Positional Encoding Cross-Entropy Gradient Descent

CourseVisit source

OpenAI Spinning Up in Deep RL

Implementation-oriented RL guide

Deep RL concepts, algorithms, exercises, failure modes, and benchmarks.

Why study it: It teaches a mature pattern: explain the idea, implement it, then test failure modes.
Borrow in spirit: Use its failure-mode habit for alignment, RLHF, and agent sections.
Use policy: Link to official docs and repo; cite; keep local code witnesses minimal and original.

RLHF Reward Hacking Process Reward Models Tree Search Reasoning

CodebaseVisit source

OpenAI Cookbook

Applied API examples

Practical examples and guides for building with modern AI APIs.

Why study it: It calibrates what useful applied notebooks feel like when someone wants to build now.
Borrow in spirit: Borrow the task-to-code directness, while grounding every recipe in the underlying math object.
Use policy: Link to examples and docs; respect repository license before reusing code.

Retrieval-Augmented Generation Structured Decoding Decoding and Sampling LLM Serving

Engineering guideVisit source

Full Stack Deep Learning

AI product engineering

Problem formulation, data, training, reproducibility, deployment, and product considerations.

Why study it: It fills the gap between model understanding and shipping AI systems responsibly.
Borrow in spirit: Use the project lifecycle as an engineering lane beside the mathematical atlas.
Use policy: Link to course pages and materials; keep local systems guidance original.

LLM Serving Retrieval-Augmented Generation Structured Decoding efficiency

Engineering guideVisit source

Made With ML

Production ML guide

Designing, developing, deploying, and iterating production ML applications.

Why study it: It brings software-engineering discipline into the learning loop.
Borrow in spirit: Turn production concerns into labs: data checks, evaluation, drift, serving, and monitoring.
Use policy: Link to lessons and repo; do not copy prose or code wholesale.

LLM Serving Knowledge Distillation Quantization Pretraining Data Mixtures

Visual essayVisit source

Distill

Interactive research communication

Visual, interactive explanations of neural networks, interpretability, and optimization.

Why study it: It is one of the clearest examples of web-native explanation as serious research communication.
Borrow in spirit: Borrow the standard of interactive clarity, not any specific visual style.
Use policy: Check article-level licenses before reuse; prefer citations and original repo-native visuals.

Representations Sparse Autoencoders Loss Landscapes Attention and Transformers

Visual essayVisit source

colah's blog

Conceptual visual essays

Neural-network intuition, topology, recurrent nets, convolutions, attention, and interpretability.

Why study it: It shows how a difficult idea can become thinkable through the right picture and sentence order.
Borrow in spirit: Use spatial metaphors carefully, then pin them to equations and failure cases.
Use policy: Link and cite; do not copy figures or prose without explicit license support.

Representations Attention and Transformers Sparse Autoencoders Vector Spaces

Visual essayVisit source

Jay Alammar visual explanations

Accessible mechanism visuals

Transformers, retrieval, embeddings, and LLM mechanisms through clear diagrams.

Why study it: It demonstrates how visual sequencing can lower the activation energy for hard architecture ideas.
Borrow in spirit: Use progressive reveal for mechanisms, then add source/equation/code rigor that blog posts often skip.
Use policy: Link to posts; use as visual pedagogy inspiration only unless reuse rights are explicit.

Attention and Transformers Retrieval-Augmented Generation Tokenization and Vocabulary Decoding and Sampling

Visual essayVisit source

Lil'Log

Research synthesis blog

Careful long-form syntheses across transformers, diffusion, RL, alignment, agents, and inference.

Why study it: It is a high bar for integrating research papers into a coherent narrative without hype.
Borrow in spirit: Borrow the literature-map discipline, then attach each claim to local concepts and labs.
Use policy: Link and cite posts; write original summaries and maintain source boundaries.

Diffusion RLHF Reward Hacking Long Context

Paper threadVisit source

Transformer Circuits

Mechanistic interpretability research thread

Reverse-engineering transformer circuits, attention heads, features, and language-model internals.

Why study it: It is the strongest model for treating model internals as exact research objects.
Borrow in spirit: Make equations, heads, features, prompts, activations, and claims first-class learning objects.
Use policy: Deep link to articles; cite; keep local interpretability demos original and bounded.

Attention and Transformers Sparse Autoencoders Representations Scaling Laws

CodebaseVisit source

TransformerLens

Mechanistic interpretability toolkit

Practical tools, tutorials, and exercises for inspecting transformer activations and circuits.

Why study it: It shows how interpretability can become a hands-on lab discipline.
Borrow in spirit: Use it as the practical bridge from concept pages to real model inspection workflows.
Use policy: Link to docs and repo; respect code license before reuse.

Attention and Transformers Sparse Autoencoders Representations Scaling Laws

CourseVisit source

Berkeley CS285 Deep Reinforcement Learning

Deep RL course

Modern RL methods, offline-to-online RL, LLM RL, and project-driven RL practice.

Why study it: It helps connect reward, feedback, exploration, and policy optimization to modern alignment practice.
Borrow in spirit: Use RL as a lab for feedback loops, failure regimes, and reward misspecification.
Use policy: Link to official materials; avoid copying assignments or lecture assets.

RLHF Reward Hacking Process Reward Models Tree Search Reasoning

CourseVisit source

Berkeley Deep Unsupervised Learning

Generative and self-supervised learning course

Deep generative models, self-supervised learning, diffusion, flows, and modern representation learning.

Why study it: It gives the generative-model track a research-course backbone.
Borrow in spirit: Turn generative families into comparable mechanisms: density, score, flow, and sample path.
Use policy: Link to official course pages; do not copy slides, assignments, or videos.

Diffusion Score Matching Flow Matching Representations

What Continuous Function must add

The advantage is connection, not ownership of the raw material.

Prerequisites are rarely attached to the exact object

Common pattern: A book may teach linear algebra well, while a paper assumes the exact vector-space move silently.

Our move: Attach prerequisite repair directly to the equation, claim, code witness, or demo state that needs it.

Math, code, and demos often drift apart

Common pattern: A course gives a derivation, a notebook gives implementation, and a blog gives intuition.

Our move: Keep intuition, symbols, tensor shapes, code, and manipulation aligned around one selected object.

Reading often happens without prediction

Common pattern: Learners consume explanations and only discover confusion when they try to build.

Our move: Ask for a prediction before reveal, then preserve the smallest useful observation.

Frontier resources age unevenly

Common pattern: Classic material is stable but incomplete; fresh blogs are useful but unevenly grounded.

Our move: Use stable foundations as anchors and frontier sources as inspectable, source-bounded objects.

Research practice is separated from learning

Common pattern: Courses teach ideas, while papers and labs require a different workflow.

Our move: Make paper mapping, source checking, code witnesses, and next experiments part of the same route.

Build order

Content first, then memory and services.

Publish the AI Lab curriculum spine as a public local page.
Deepen one flagship route: Attention -> KV cache -> long context -> serving.
Turn the top source families into concept-connected resource cards with license notes.
Add diagnostic entry questions that choose a route without requiring account memory.
Only then connect durable learner memory, AI tutor, subscriptions, and object rooms.