What Is Machine Learning?
Some problems cannot be solved by writing rules, no matter how many you write. Stack rules one at a time and watch each one buy you less than the last.
Machine learning fits rules from examples. Instead of telling a computer every rule, you show it examples and let it find a pattern. This lesson explains when that approach works and when it does not.
A problem you can write rules for
Ordinary programming is telling a computer exactly what to do. If you want to know whether someone can vote, you write the rule:
def can_vote(person): return person.age >= 18 and person.is_registeredThis is a good program. It is fast, it is exactly right, anyone can read it, and when the law changes you change one line. Do not use machine learning for this. If you can write the rule down, write the rule down.
A problem you cannot
Now try: is this message spam?
You start writing rules. Lots of links is suspicious — unless it is a newsletter. ALL CAPS is suspicious — unless it is a short excited message from a friend. The word “free” is suspicious — unless the email is about free shipping on something you ordered.
Every rule you write needs an exception, and the exceptions need exceptions. Try it below: each step adds the single best rule that can be added, chosen automatically, so you are seeing hand-written rules at their absolute best.
240 messages, plotted by length against how many links they contain. Filled circles are spam. Each rule you add covers a rectangle of the chart; rust rings mark the messages still being got wrong.
- spam (118)
- not spam (122)
- what a model learns instead
no rules yet
With no rules the filter calls everything safe. It catches nothing, and it has flagged nothing — so there is no precision to report at all.
Accuracy
0.51still wrong: 118 of 240 · a learned line gets 0.96
Precision of your rules
—nothing flagged yet — there is no ratio to report
What each rule bought
What the machine does instead
Two things matter here. The second matters more.
First, the returns collapse. The first rule buys you thirty percentage points. By rule six the gains are down to fractions of a point, and rules seven and eight buy literally nothing — there is no rectangle left to add. The effort per unit of improvement keeps climbing.
Second, look at the shape. Each rule is a rectangle, because that is what an if-statement about two numbers draws. But the real boundary between spam and not-spam runs diagonally — three links in a short message is suspicious, three links in a long newsletter is not. A staircase of rectangles can get closer and closer to a diagonal and never become one.
The dashed line is what a model learns from the same data. It is one line rather than eight rules, nobody wrote it, and it does better than the rules ever manage.
What this costs you
This trade is not free. Know the price before you start.
You need examples, and lots of them. The rule for voting needed nobody’s data. A spam filter needs thousands of messages that somebody already labelled.
You lose the ability to read it. You can read age >= 18 and know exactly what it does. A learned model is a pile of numbers, and explaining any single decision is its own hard problem.
It is never exactly right. Notice that even the learned line does not get everything — some of these messages are ambiguous, and no method recovers those. Machine learning trades certainty for coverage.
Key takeaways
- Ordinary programming: you write the rules. Machine learning: you supply examples and the rules are worked out.
- If you can write the rule down, write the rule down. Rules are faster, clearer and exactly right.
- Hand-written rules do not fail all at once — each one buys less than the last until they buy nothing.
- If-statements draw rectangles. When the real boundary is a diagonal, a staircase can approach it but never become it.
- The trade costs you data, readability, and certainty. Make it when the exceptions would never end.
Quick check
Answer these to unlock the next chapter — 3 of 4 to pass. You can retake it anytime.
Answer every question to check.
Make a free account to read on
Every chapter is free — an account is how your progress, XP, and streak follow you from your laptop to your phone, and how you show up on the leaderboard. No payment, no trial.