Domain Neighborhood
Optimization
How we train models: gradients, learning rates, curvature, and the practical tricks that make deep nets converge.
Start here. Predict once, then carry the invariant forward.
θ_t∇L(θ)updateθ_{t+1}Learner lensWhat makes this domain feel navigable?
Take this lensStabilize the first mechanism, make one prediction, then move one node forward.
Recommended Route
Start here, then follow the prerequisites forward.
This sequence is ordered for learning rather than inventory. Published notebooks with an unavailable prerequisite—or a same-domain route step that depends on one—are labeled in the full inventory instead of being presented as ready steps.
- 01Gradient Descent
Gradient descent turns local slope information into an iterative update rule for reducing a loss.
12 mincodedemoafter DerivativesCheck Derivatives first if the symbols feel slippery.
All Published Notebooks
Browse the territory.
Gradient Descent
Gradient descent turns local slope information into an iterative update rule for reducing a loss.
Adam Optimizer
Adam is an adaptive optimizer that combines momentum (EMA of gradients) with per-parameter RMS normalization (EMA of squared gradients).
Loss Landscapes, Sharpness & Flat Minima
How 2D loss slices, Hessian curvature, SAM-style neighborhood loss, and a toy 2/eta stability line expose local sensitivity during optimization.
Learning Rate Schedules: Warmup, Decay & Cycling
Schedule shapes that change the scalar learning-rate scale over training, with sourced CLR/range-test and SGDR cosine-restart examples plus caveated warmup/decay teaching patterns.
3 published notebooks are readable now but held out of the recommended route until the named prerequisite notebooks are published.