Transfer Learning

Training a serious image classifier from a random start wants somewhere in the range of a million labelled photos, spread reasonably evenly across your classes, and several days on a bank of GPUs to converge. ImageNet, the dataset most of these networks were first proven on, has about 1.2 million training images across 1,000 categories.

A student project, or a small team building a tool for one specific problem, has none of that. You might have four hundred photos of a specific product defect, gathered over a month, with no budget for more. Training the last lesson’s network on four hundred photos from scratch does not produce a worse classifier. It produces one that has memorised four hundred photos and generalises to nothing.

A network trained on a broad photo dataset — a million images spanning cars, dogs, kitchens, mountains — has already been through everything the last lesson described. Its early layers settled on edge and colour detectors. Its middle layers settled on corners, textures, and simple parts. None of that is specific to the exact 1,000 categories it was scored on.

An edge detector that helps recognise a bicycle wheel helps just as much recognising a clock face. A texture detector trained to spot fur helps with carpet. The knowledge sitting in those layers is closer to how to see than how to name these 1,000 things, and how to see transfers.

The mechanic is direct. Take a network already trained on the broad dataset. Freeze every layer except the last — literally stop their weights from updating — and replace the final classification layer, the one that outputs one of the original 1,000 categories, with a new one that outputs your handful of categories instead.

Retrain only that new final layer, on your four hundred photos. You are not asking the network to relearn what an edge is or what a wheel looks like; those layers are untouched. You are only asking it to learn a new mapping from features it can already extract onto your specific labels, which is a far smaller thing to learn and needs far fewer examples to learn it — dozens or low hundreds of photos per class is often enough, not a million.

Freezing everything is the cautious end of a dial. With a few thousand photos rather than a few hundred, a common next step is fine-tuning: once the new head has trained, unfreeze the last few layers of the body as well and keep training with a small learning rate, so the later, more specific features can adjust to your images without wiping out what the early layers already know. With very little data, leave the body frozen; there is not enough to adjust it without overfitting.

This is not a universal fix, and the limit is exactly where you would guess from how it works. Transfer learning helps when your new photos are visually similar in kind to what the network originally saw — natural photos, ordinary lighting, everyday objects at everyday distances.

It helps far less carrying that same network onto a chest X-ray or a satellite image. Those images do not have the same edges, textures, or colour statistics that produced the original layers; an X-ray has no colour at all, and a satellite photo has no sense of a horizon or a gravity-consistent orientation. The features a natural-photo network learned are a poor match for what actually distinguishes a healthy lung scan from a fractured rib, so freezing them buys you far less.