Bounding Boxes and IoU
A predicted box that is "close" to the right one needs a number, not a shrug, and that number is Intersection over Union. Drag two boxes apart and watch a plausible-looking overlap score fall below 0.5.
Worth reading first: Object Detection
Two predicted boxes can both look roughly right to your eye and be numerically nowhere near each other. Intersection over Union is the number that ends the argument.
Two boxes and a disagreement
Draw a box around a dog by hand and someone else draws their own box around the same dog, and the two will not match exactly. One runs a little wide, the other clips an ear. Both look “about right”.
“About right” is not something a benchmark can grade. Comparing a detector’s predicted box against the true box needs a single number, computed the same way every time, that turns two rectangles into one verdict.
The overlap, divided by the union
Intersection over Union, IoU, is exactly what its name says: take the area where the two boxes overlap, and divide it by the total area either box covers.
Take a concrete pair. A true box covering 12,000 square pixels and a predicted box covering 10,000 square pixels overlap in a region of 8,000 square pixels. The union — everything covered by at least one of the two boxes — is 12,000 + 10,000 − 8,000 = 14,000 (subtracting the overlap once, so it is not counted twice). IoU is 8,000 divided by 14,000, which is about 0.57.
Two boxes sitting exactly on top of each other score 1.0. Two boxes that do not touch at all score 0. Everything a detector actually produces lands somewhere in between, and the interactive below lets you watch that number move.
The solid box is the ground truth. The dashed box is a prediction you can move. The shaded region is where the two overlap.
Ground truth and predicted box overlap with an IoU of 0.59, which counts as correct at a 0.5 threshold.
IoU
0.59At a 0.5 threshold, this counts as a correct detection.
What counts as a correct detection
IoU is what turns “roughly right” into a hard yes or no. A detection only counts as correct if its IoU against the true box clears a chosen threshold — everything at or above it is a hit, everything below is a miss, with nothing in between.
Nudge the predicted box in the interactive above until the number crosses 0.5, and notice there is no visual moment where the box “suddenly” becomes wrong. The picture changes continuously. The verdict does not — it flips the instant the number crosses the line.
Why 0.5 is a choice, not a law
0.5 is the threshold you will see quoted most often, but it is a convention, not a property of geometry. A benchmark can and does choose differently — some standard evaluations report accuracy at 0.75, a noticeably stricter bar, and some report a whole curve of scores across every threshold from 0.5 to 0.95 rather than commit to one.
Raising the threshold does not just get “harder” in the abstract. It can flip a specific detection from correct to wrong without the predicted box moving a single pixel — the box stays exactly where it was, and the number needed to call it right went up underneath it.
Key takeaways
- Two boxes can both look right by eye while scoring very differently — IoU exists to replace that judgement with one number.
- IoU is the overlap area divided by the union area: identical boxes score 1.0, boxes that do not touch score 0.
- A detection only counts as correct once its IoU against the true box clears a chosen threshold — there is no partial credit at the boundary.
- 0.5 is the threshold quoted most often, but it is a convention. Stricter benchmarks require 0.75 or report accuracy across a whole range of thresholds.
- Raising the threshold can turn a previously correct detection into a wrong one without the predicted box changing at all.
Quick check
Answer these to unlock the next chapter — 3 of 4 to pass. You can retake it anytime.
Answer every question to check.
Make a free account to read on
Every chapter is free — an account is how your progress, XP, and streak follow you from your laptop to your phone, and how you show up on the leaderboard. No payment, no trial.