Lesson · about 8 minutes · one exact model
When an average hides a failing group
A classifier can be right on 94.5% of photos and wrong on most of a rare group. Move the training mix and watch it happen; predict what balancing does if you like, check it, and leave with a question for real data.
Builds on cross-entropy (the loss being minimised), the dot product (the boundary is where w·x + b = 0) and representations (what a model’s “features” are). Your place here is kept while you look.
The model · training mix 95% · shape clarity 1.00 · trained as collected
Average, weighted like training: 94.5%Worst group: 39.2%
Accuracy on new photos of each group, exact for this model.
The boundary tilts 53° from vertical (0° ignores the background; 90° uses only it). In what the fit sees, the background goes with the bird 95% of the time, and it is cleaner than the shape, so fitting well rewards it, and the rare groups, where it points the wrong way, pay.
A shortcut that pays in training
Each dot is a training photo: its shape (across) says which bird it is, its background (up) says land or water. The shape is noisy; the background is clean. In training, most waterbirds are on water and most landbirds on land.
Computed for this modelThe readout that best fits this training data is right 94.5% of the time on photos mixed like training, and 39.2% on waterbirds on land. Its boundary tilts 53° from vertical: the more it tilts, the more it uses the background.