How a Model Learns

Say you want to predict how much fuel a car trip will use. You have thirty-six past trips: how far each went, and how many litres it burned.

The simplest useful model is one number — litres per 100 kilometres. Pick that number and you can predict any trip: multiply, done. A 150 km trip at 8 L/100km is 12 litres.

Before you can improve a guess you need to score it. For each past trip, compare what the model predicts against what actually happened. The difference is the miss.

Add up all the misses and you have a single number describing how wrong the whole model is. That number is the only thing the learning process ever looks at.

One detail matters now: misses are usually squared before they are added. Squaring makes every miss positive, so overshooting by two does not cancel out undershooting by two — and it makes one large miss count for more than several small ones, which is usually what you want.

Now try it. Drag the slider to change the one number the model has, and watch two things at once: the rust sticks in the top chart, and the dot moving along the curve underneath.

The curve underneath is the important part: it is the total error at every setting you could have picked. The dots are scattered and messy. The curve is a smooth bowl.

That smoothness is the entire reason a machine can do this without understanding anything about cars. It does not need to know what fuel is. It only needs to be able to ask “is the error smaller if I nudge this number up, or down?” and then step that way. Thousands of times, very quickly.

You can do the whole search in a few lines. Score a handful of settings, then try every setting from 4 to 11 in steps of 0.01 and keep whichever leaves the least total squared error.

import numpy as np rng = np.random.default_rng(1)km = rng.uniform(20, 300, 36)                     # 36 past tripslitres = km * 7.5 / 100 + rng.normal(0, 1.5, 36)  # really about 7.5 L/100km def total_error(rate):    predicted = km * rate / 100    misses = predicted - litres    return np.sum(misses ** 2)                     # squared, then added up for rate in [5, 6, 7, 8, 9, 10]:    print(f"{rate} L/100km -> total squared error {total_error(rate):8.1f}") # Try every setting from 4 to 11 in small steps and keep the lowest.rates = np.arange(4, 11, 0.01)best = rates[np.argmin([total_error(r) for r in rates])]print(f"best setting: {best:.2f} L/100km")

The errors fall and then rise again — the bowl, as numbers — and the lowest lands at 7.40 L/100km. The trips were generated at 7.5, so even the best setting is not the true one: the noise in thirty-six trips pulls it slightly off.

Real models have millions of numbers instead of one, and the bowl becomes a landscape in millions of dimensions that nobody can picture. But the move is identical, and you have now done it by hand.