Domain
Probability
Events, random variables, expectations and the distributions used in machine learning.
Preview the recommended route
Start here. Make one prediction, then note what carries over to the next notebook.
Xp(x)E[·]uncertaintyLearnerWhich notebook should I read first?
Open the first notebookRead the first notebook, make one prediction in its demo, then go on to the next.
Recommended route
Start here, then follow the prerequisites forward.
Read these in order: each notebook comes after the notebooks in this domain that it builds on, and the label beside it gives its level. A prerequisite with no page here yet is marked as outside reading. A notebook is left out of the route only when a prerequisite has neither a page here nor a named outside reading; the full list below marks it.
- 01Probability Basics
Events are subsets of a sample space, and probabilities obey a few axioms; from there you get conditional probability, independence, and Bayes' rule.
Introductory12 mincodedemono prerequisites - 02Random Variables
A random variable is a function from outcomes to numbers; its distribution lets you compute expectations, variances, and likelihoods.
Introductory14 mincodedemoafter Probability BasicsRandom Variables builds directly on Probability Basics.
- 03Distributions
A distribution is the law of a random variable: it says how probability mass or density lands on the values the variable can take.
Introductory15 mincodedemoafter Random VariablesDistributions builds directly on Random Variables.
- 04Maximum Likelihood
Maximum likelihood fits parameters by making the observed data most probable; for classifiers it becomes negative log-likelihood, cross-entropy, and a KL fit to the empirical distribution.
Intermediate18 mincodedemoafter Distributions, DerivativesMaximum Likelihood builds directly on Distributions.
- 05Cross-Entropy
Cross-entropy is the target-weighted surprise of a model distribution; in deep learning it is the bridge from likelihood to a differentiable training loss.
Intermediate16 mincodedemoafter Maximum LikelihoodCross-Entropy builds directly on Maximum Likelihood.
- 06Bayesian Inference
Bayesian inference updates a prior distribution over unknowns into a posterior by multiplying by the likelihood and normalizing.
Intermediate17 mincodedemoafter Distributions, Maximum Likelihood
All notebooks
Every published notebook in this domain.
Probability Basics
Events are subsets of a sample space, and probabilities obey a few axioms; from there you get conditional probability, independence, and Bayes' rule.
Random Variables
A random variable is a function from outcomes to numbers; its distribution lets you compute expectations, variances, and likelihoods.
Distributions
A distribution is the law of a random variable: it says how probability mass or density lands on the values the variable can take.
Maximum Likelihood
Maximum likelihood fits parameters by making the observed data most probable; for classifiers it becomes negative log-likelihood, cross-entropy, and a KL fit to the empirical distribution.
Cross-Entropy
Cross-entropy is the target-weighted surprise of a model distribution; in deep learning it is the bridge from likelihood to a differentiable training loss.
Bayesian Inference
Bayesian inference updates a prior distribution over unknowns into a posterior by multiplying by the likelihood and normalizing.