Classification vs Regression
Categories or numbers? The line between them is a choice you make — and chopping a number into categories can make your accuracy look better while the model gets less useful.
Read first: How a Model Learns
Almost every supervised problem is one of two shapes, and telling them apart is easy. What is less obvious — and much more useful — is that the line between them is a choice you make, and moving it changes what your accuracy score is even measuring.
Two kinds of question
Classification
The answer is one of a fixed set of categories. Spam or not spam. Cooked or doughy. Which of ten digits. You measure it with accuracy — how often the named category was right.
Regression
The answer is a number on a scale. Minutes. Price. Temperature tomorrow. You measure it with average miss — how far off you typically were, in the units of the thing itself.
That is the whole distinction, and you can usually tell within a sentence of hearing the problem. “Will this customer cancel?” is classification. “How much will they spend?” is regression.
The same model, both ways
A number can always be chopped into categories. “How many minutes will this delivery take?” becomes “is it fast or slow?” the moment you pick a dividing line — and now the same problem is a classification.
Below, one model predicts delivery time from distance. The only thing the slider changes is how many categories that prediction gets rounded into.
180 deliveries, plotted by distance against how long they took. One model throughout — only the shape of its answer changes. The bands are the categories; the step line is what the classifier actually says; rust dots are the ones it puts in the wrong band.
- what the classifier says
- the same model as a plain number
- wrong category
2 categories, each 32 minutes wide
This is a classifier. It is right most of the time — and when it says “slow” it means somewhere in a half-hour window. The customer cannot use that.
Accuracy
0.84152 of 180 landed in the named category
Average miss
9.2plain number instead: 5.9 minutes off
What the model says about a 9 km delivery
- as 2 categories → “slow” (37–69 minutes)
- as 20 categories → “40 to 44 minutes”
- as a plain number → 40.6 minutes
Accuracy is not comparable
Drag from two buckets to twenty and watch the two numbers move in opposite directions. Accuracy falls off a cliff. The average miss — how wrong the answer actually is, in minutes — gets better.
At two categories the model is right most of the time, and telling a customer their delivery is “slow” when that covers a half-hour window is nearly useless. At twenty categories it is right far less often, and its answer is far more use.
Try it with your own numbers. One regression model predicts delivery minutes; then the very same predictions are rounded into 2, 5 and 20 equal-width buckets and scored as categories.
import numpy as npfrom sklearn.linear_model import LinearRegression rng = np.random.default_rng(2)km = rng.uniform(1, 15, 300)minutes = 10 + 3 * km + rng.normal(0, 6, 300) # delivery time model = LinearRegression().fit(km.reshape(-1, 1), minutes)predicted = model.predict(km.reshape(-1, 1))print(f"average miss: {np.mean(np.abs(predicted - minutes)):.1f} minutes") # Now round the same predictions into equal-width buckets and# score them as categories instead.for buckets in [2, 5, 20]: edges = np.linspace(minutes.min(), minutes.max(), buckets + 1)[1:-1] right = np.digitize(predicted, edges) == np.digitize(minutes, edges) print(f"{buckets:2} buckets: accuracy {right.mean():.0%}")The average miss is 4.7 minutes, and accuracy falls from 88% to 60% to 19% as the buckets narrow. The predictions never changed. Only the question did.
Choosing the shape of your answer
So which should you build? Ask what the person receiving the answer will actually do with it.
If they need to act differently depending on the answer — approve or decline, urgent or routine — categories are the right shape, and you should pick the boundaries to match those actions rather than splitting the range evenly.
If they need to plan against a quantity — how many drivers, how much stock, when to leave — give them the number. Notice in the chart that no amount of bucketing ever quite catches the plain number: rounding an answer into a band can only lose information, never add it.
Key takeaways
- Classification predicts a category and is scored with accuracy. Regression predicts a number and is scored with average miss.
- Any number can be turned into categories by choosing dividing lines — so the distinction is partly your decision.
- Chopping a number into more categories makes accuracy fall while the answer gets more useful.
- Accuracy cannot be compared across problems with different numbers of categories. It is not a universal score.
- Choose categories when someone will act differently on each one; choose a number when they need to plan against it.
Before you move on
Quick check
Answer these to unlock the next chapter: 3 of 4 to pass. You can retake it anytime.
Make a free account to read on
Every chapter is free — an account lets your course progress follow you from your laptop to your phone. No payment, no trial.