By the end of this practical you will be able to:
A grower walking through a tomato greenhouse can spot a sick plant at a glance. Doing that at the scale of a whole breeding programme, thousands of plants, several times per season, is a different problem. This is where image-based phenotyping comes in: a camera on a drone, a robot or a phone takes the pictures, and a model decides which leaves look diseased.
In this practical you will train such a model yourself. It runs in your browser, on real photographs of tomato leaves. But the real lesson is not "how do I train a model". That turns out to be the easy part, about ten lines of code. The lesson is how do I find out whether I should trust it. You will meet a model with 90% accuracy that is completely worthless, and a model that learns the wrong thing so convincingly that only careful evaluation reveals it.
In the mechanistic modelling practical you simulated a gene regulatory network. There, we supplied the biology: "ABA activates ABF2", "ANAC019 represses ICS1". Those rules were the input, written down by researchers after years of experiments, and the computer only worked out their consequences over time.
Deep learning turns this around completely. Nobody tells the model what a diseased leaf looks like. Nobody writes down "brown lesions with a yellow halo mean Septoria leaf spot". Instead we hand over a pile of photographs, each labelled healthy or infected, and the model works out the rules itself by looking for whatever distinguishes one pile from the other.
We use the PlantVillage dataset: photographs of single tomato leaves against a grey background, each one labelled by a plant pathologist. There is one healthy category and nine disease categories: bacterial spot, early blight, late blight, leaf mold, Septoria leaf spot, spider mites, target spot, yellow leaf curl virus and mosaic virus.
We are going to group all nine diseases together into a single infected class, so the model has to answer one binary question: healthy, or not? We could equally well have trained it to name the specific disease. That is a nine-way problem, and a harder one.
Every image has been shrunk to 64 × 64 pixels, which is the size the model works with.
This is the single most important idea in this section, and it is worth slowing down for. The model has no eyes. It never sees a leaf, a lesion or a colour. What arrives at its input is a grid of numbers, and nothing else.
Each of the 64 × 64 pixels stores three numbers: how much red, how much green and how much blue that pixel contains. A camera records those as whole numbers from 0 to 255, and before feeding them to the model we divide by 255 so everything sits between 0 and 1.
Hover over the leaf below. The panel on the right shows you the actual numbers stored in the small square you are pointing at. This is, quite literally, the model's entire view of the world.
Move your mouse over the leaf. The white square marks the 6×6 patch shown on the right.
Before we train anything, we have to hold some data back. This is one of the most important habits in the whole of machine learning, so it is worth being precise about why.
A model that has already seen a photo can score well on it simply by having memorised it. Memorising tells you nothing about the next leaf, and the next leaf is the entire point: you want a model that works on plants it has never encountered. So we cut the dataset in two. The model learns from the training set, and is judged on the test set (also called the validation set), which it never learns from. Any number you quote about a model's performance should come from data it did not train on.
Below you control how big each half is. Watch what the split does to the number of images the model gets to learn from, and to the number of images your performance estimate is based on.
Before you build your own, let me offer you mine. I have developed a classifier and it reaches roughly 90% accuracy on the test set. Ninety percent, on a real biological problem, with no effort on your part. Press the button and see for yourself.
Answer those before scrolling on. The rest of this step works through what you should have asked for.
Accuracy is just the fraction of images the model gets right. That single number hides something important, and the something is the class distribution: how many of each kind of leaf there are in the first place. Drag the slider to change how many healthy leaves are in the dataset, and watch what happens to my model's accuracy.
Overall accuracy averages over both classes, so a model can be excellent on one and hopeless on the other without the single number ever showing it. Split the score by class and look again.
The confusion matrix lays out all four possible outcomes instead of collapsing them into one number. Each leaf is either truly healthy or truly infected (the two rows), and the model either called it healthy or infected (the two columns). Every leaf lands in exactly one of the four boxes. The two boxes on the diagonal are the correct answers, and the two off it are the two different ways of being wrong.
Confusion matrix on the test set
There is one more angle, and it takes a little more explaining, so let's build it up slowly.
A classifier does not really answer "healthy" or "infected". It produces a score, and we compare that score against a cut-off to get an answer. Change the cut-off and you get a different set of answers from exactly the same model. So judging a model at one cut-off only tells you about that one choice.
The ROC curve gets around this by trying every possible cut-off and plotting two things at each one:
Both are things you want to trade off against each other. Catching more infections always means raising more false alarms, and the curve shows you the exchange rate. A model that is good at telling the classes apart can catch a lot of infections while raising few false alarms, so its curve bulges towards the top left. A model with no ability to discriminate can only buy catches by accepting an equal share of false alarms, so its curve runs along the diagonal. That diagonal is drawn dashed on the plot below as a reference for "no better than guessing".
The area under the curve, the AUROC, compresses that into one number: 1.0 is a perfect ranker, 0.5 is a coin flip.
ROC curve on the test set
More background on these two: thresholding and the confusion matrix and ROC and AUC.
Now let's train an actual neural network and see whether it can beat that. What follows is not a demonstration or a replay. The network really is built and trained in your browser, on your machine, from the images above. That is also why it takes a few seconds.
You do not need to know the internals to follow what happens. Training works like this: the network starts with random settings and therefore guesses randomly. We show it a training image, it produces a number between 0 and 1, and we compare that with the true label. If it was wrong, every internal setting is nudged a little in the direction that would have made the answer better. Then the next image. One full pass through all the training images is called an epoch.
The model we use is a small convolutional network, the variety designed for images, which looks for local patterns like edges, spots and patches of colour rather than treating each of the 12,288 numbers as unrelated.
Your dataset is currently about 10% healthy, because that is what step 3 needed in order to make its point. That is a bad setting to train on. If nine out of ten training images are infected, the laziest thing the network can do is answer "infected" almost every time, and that already scores well. You will have built your own version of the model you just rejected.
So before training, raise the healthy fraction. This is the same slider as in step 3, repeated here for convenience.
More epochs means more passes over the data and a better-fitted model, up to the point where it starts memorising. Watch the curves rather than trusting a fixed number: you want to train until the test curve stops improving.
Loss, how wrong the model is. Lower is better.
Accuracy over the same run. Higher is better.
The network does not answer "healthy" or "infected". Its final output is a single number between 0 and 1 for each image. Here are its raw outputs for ten test images:
The number under each leaf is the network's raw output; the line below it is the true label.
To get from a number between 0 and 1 to an actual decision, we need a threshold: above it we call the leaf infected, below it we call it healthy. Nothing forces that threshold to be 0.5. Choosing it is a decision about which mistake you would rather make, and it is yours to make, not the model's.
Numbers tell you how often the model is wrong. Looking at the images tells you why. This is one of the most useful habits you can pick up: whenever a model disappoints you, go and look at the cases it got wrong. Each group below corresponds to one box of the confusion matrix, and the groups re-sort themselves as you move the threshold.
Here is a story that happens more often than anyone would like. A grower photographs the healthy part of the field in the morning and walks over to the infected part later in the day, when the light has changed. Nobody notices; the photos all look fine. But every healthy photo is now slightly darker than every infected one.
Let's simulate exactly that. In the training set we darken every healthy leaf a little. And because the model will be used by other growers whose habits differ, in the test set we darken the infected ones instead. Nothing about the leaves themselves has changed. Only the lighting has.
Training set, healthy leaves darkened
Test set, infected leaves darkened
Loss. Watch the two lines pull apart.
Accuracy on the training set against the test set.
Confusion matrix on the test set (threshold 0.5)
The model latched onto brightness because, in its training set, brightness was a perfectly reliable clue. One way to take that clue away is image augmentation. Before each pass we randomly mess with the training images: mirror them, brighten or darken them, nudge their contrast. The labels stay the same.
Crucially, this is re-rolled every epoch, so the model never sees the same version of an image twice. Press the button below a few times to see different random versions of the same training leaves.
The same confounded training leaves as in step 7, randomly flipped, brightened/darkened and contrast-adjusted.
Augmented images are harder to learn from, so this model usually needs more epochs than the others before it catches up. If the improvement looks disappointing, train it for longer before concluding that augmentation did not work.
Loss. Compare the gap with the one in step 7.
Accuracy on the training set against the test set.
The same confusion matrix and threshold slider as in step 5, this time on the augmented model. Compare the shape of the errors against step 7. A model that has stopped leaning on brightness should spread its mistakes around rather than piling them all into one box.
Train the model above to see its confusion matrix.
Test-set ROC curves compared. Train the step 7 model first so there is something to compare against.