Deep Learning on Leaf Images: Answer Model

• Mechanistic vs. data-driven. A question like "how does a longer drought shift the trade-off between drought tolerance and pathogen defence?" calls for a mechanistic model: you are asking why, you already have regulatory knowledge to encode, and you want to reason about interventions you have never tried. A question like "which of these 40,000 field photographs show disease?" calls for a data-driven model: nobody can write the rule down, but labelled examples are cheap. The deciding factors are whether you have mechanism you trust or data you trust, and whether you need an explanation or a decision.
• Both at once. It is a strength because no theory is required, so it works in areas where our understanding is incomplete, and it can pick up subtle cues a human could never put into words. It is a weakness because the model has no notion of what is biologically relevant: it cannot tell a causal cue (a lesion) from an incidental one (the lighting, the background, the camera). It cannot explain itself, cannot be checked against known biology, and will apply a nonsensical rule with complete confidence. Both statements are true simultaneously, which is what the rest of this practical is about.

• What the numbers are. Each individual number is the intensity of one colour channel at one pixel position. There are three per pixel because any colour can be reconstructed by mixing red, green and blue light, so the camera records how much of each is present. Cameras store these as whole numbers from 0 (none) to 255 (maximum); we divide by 255 so every input sits between 0 and 1, a range in which neural networks train much more stably. One leaf is therefore 64 × 64 × 3 = 12,288 numbers, plus a single label of 0 or 1.
• What shrinking costs. We throw away fine detail: small or early lesions, spore structures, fine vein texture, anything smaller than roughly a pixel at the new size. We do it because compute and memory scale with the number of pixels, and because fewer inputs means fewer things the model can latch onto, which matters a lot when you only have a few hundred training images. It is a trade-off: too small and the symptom itself disappears. At 64 × 64 the larger blotches and colour changes still survive.
• The uniform background: both. Helpful, because it removes a large source of irrelevant variation and lets the model concentrate on the leaf. Harmful, because it is unrealistically easy, a model trained on studio photographs against grey card has never seen soil, neighbouring leaves, shadows or direct sunlight, and will very likely fall over on a real field photo. It also means anything systematic about the setup rather than the leaf becomes available as a shortcut, which is precisely the trap in step 7.

• Why split. To measure generalisation. A model can score perfectly on its training images simply by memorising them, which tells you nothing. Only performance on data it never learned from estimates how it will behave on the next leaf you show it, and that is the only thing you actually care about.
• What to consider. Keep the class distribution representative in both halves, ideally stratifying so the proportions match. Leave the test set large enough for the estimate to be stable, and the training set large enough to learn from. Make sure near-duplicates cannot straddle the split, such as the same plant, the same leaf photographed twice, or the same field session. Split along the axis you actually want to generalise over: if the model must work on a new farm, hold out whole farms rather than random images. And do not tune your model repeatedly against the test set, or it quietly stops being held out.
• The same plant in both halves. Two photos of one plant are near-identical: same leaf shape, same lighting, same disease stage, same camera. If one is in training, the model can recognise its twin in the test set without having learned anything general, so your test score partly measures memorisation and comes out optimistically biased. This is called data leakage, and it is one of the most common reasons a model that looked excellent in development disappoints in the field.
• Which number to quote. The test-set number, always. The training number describes how well the model reproduces answers it was explicitly fitted to, so it is not evidence about anything unseen. Quoting it is not just optimistic, it is measuring the wrong thing: a model that had simply memorised every training image would score 100% there while being useless in a greenhouse.

• Do you trust it? You should not, and the honest answer at this point is that you do not yet have enough information to say. That is the real lesson: a single accuracy figure is not evidence, whatever its value.
• What to ask for. A good list would include: how many of each class are there, and what does the trivial "always guess the most common class" baseline score? How does the model do on each class separately? What kinds of mistakes does it make, in a confusion matrix? How well does it rank cases, via AUROC? What does it actually predict on a handful of images I can look at? How was the test set constructed, and could it leak? If you asked for most of these, you were already thinking like a sceptic. The rest of this step works through them.

• Does the distribution change your view? Completely. About 90% of these leaves are infected, so the rule "always say infected" scores about 90% by construction. The meaningful comparison for any model is against that majority-class baseline, not against 50%. Move the slider to 50% healthy and the same model drops to about 0.50, which shows the score was a property of the dataset rather than of the model.
• What the two plots show. The confusion matrix is a table of true label against predicted label at one chosen threshold, splitting outcomes into true negatives, false positives, false negatives and true positives. It shows which mistakes are made, not just how many. Here the entire "predicted healthy" column is empty, because the model never predicts healthy. The ROC curve plots the true positive rate against the false positive rate across every possible threshold, so it describes the quality of the model's ranking independently of where you cut. Its area, the AUROC, is 1.0 for a perfect ranker and 0.5 for one carrying no information. Here the curve lies on the diagonal.
• Useful predictions? None whatsoever. It never identifies a single healthy leaf, its accuracy on the healthy class is 0.00, and its AUROC of 0.50 says it cannot separate the classes at all. It contains no information about its input: you would get identical output with the lens cap on.
• The rule in one sentence: "always answer infected", regardless of the image. (In the notebook this is MagicModel, whose predict literally returns an array of ones.)
• Looking back at your list. The thing most people miss first time is the class distribution, because accuracy feels like it should already account for it. Everything else here follows from that one question. If you would still deploy this model, ask yourself what it would do in a greenhouse where most plants are healthy: it would flag every single one.

• What the outputs are. A single number between 0 and 1 per image, produced by the final sigmoid. It expresses how strongly the network leans towards class 1 (infected): near 1 means confident infected, near 0 confident healthy. It is often loosely called a probability, but it only deserves that name if you have checked that it is calibrated, that among all images scored 0.7, roughly 70% really are infected.
• A score near 0.5. The model is undecided: whatever it extracted from the image does not clearly favour either class. In practice you would not want an automatic decision here at all. A sensible deployed system routes low-confidence cases to a human rather than forcing a call, which is often more valuable than squeezing out another percent of accuracy.
• More epochs. The training loss keeps falling and training accuracy keeps climbing, while the test curves flatten off and eventually turn the wrong way. The gap between the solid and dashed lines is overfitting made visible: it measures how much of the model's apparent skill is memorisation of these particular images rather than something that transfers. A widening gap is the signal to stop training, get more data, or regularise.

• Which matrix is more useful? The test one. The training matrix shows how well the model reproduces answers it was explicitly fitted to, which is almost always flattering. Only the test matrix estimates behaviour on new leaves. That said, the training matrix is still worth a look, because comparing the two is how you spot overfitting.
• What the threshold does. It moves cases between the columns. Raising it makes the model more reluctant to say "infected": true positives fall, false negatives rise, false positives fall, true negatives rise. Lowering it does the reverse. The row totals never change, because those are the true labels. So recall and precision move in opposite directions and accuracy has an optimum somewhere in between.
• Prefer the neural network? Yes, but not because its accuracy number is bigger. The reason is that it actually discriminates: an AUROC well above 0.5 means it genuinely ranks infected leaves above healthy ones, so it carries information about the image. It also identifies healthy leaves, which the previous model never did, and it gives you a threshold worth tuning, a meaningless notion for a model whose output is constant.
• The quarantine station. There, a missed infection is the expensive error, so you lower the threshold and call things infected on weaker evidence. That raises recall. What it costs is precision: more false alarms, more healthy consignments delayed, inspected or destroyed. Note that this is a policy decision about which error you can better afford, not a modelling decision, the model is the same either way.

• The false negatives. They tend to be leaves where the symptoms are small, sparse or early-stage, or diseases that simply look close to healthy at 64 × 64, faint viral mottling, or a handful of tiny spots. Some are just blurry or awkwardly lit. Grouping them by disease is informative: if one category dominates, that is a concrete instruction about which data to collect next.
• Moving the threshold. Raise it and false negatives grow while false positives shrink; lower it and the reverse. You cannot empty both at once unless the score distributions of the two classes do not overlap at all, that is, unless the model is perfect. That overlap is the model's real limitation; the threshold only lets you choose where within it you want to stand.
• The next 200 leaves. Collect more of what it gets wrong, not more of what it already handles. Concretely: more healthy leaves, since they are the minority class and carry most of the relative error, and more of whichever disease dominates the false negatives, specifically mild and early-stage cases rather than more textbook-obvious ones. Adding easy examples of a class the model already nails buys you almost nothing.

The prediction to make: training performance will look excellent, quite possibly better than before, because brightness has become a perfectly reliable clue within the training set. Test performance will be dreadful, most likely worse than guessing, because in the test set that same clue points the wrong way.

If your instinct was "it should be fine, the leaves themselves have not changed", that is exactly the intuition this step is designed to break. The model does not see leaves. It sees numbers, and we changed the numbers.

• The performance. Training accuracy is high, test accuracy is poor, and the test AUROC lands around or below 0.5. Note carefully that the model has not failed to learn. It has learned extremely well, it has just learned the wrong thing.
• The explanation. In the training data every healthy leaf was darker. Brightness is a far simpler and more reliable pattern than lesion shape or texture, so gradient descent found it first and leaned on it: "dark means healthy". There was never any pressure to look at the disease, because the shortcut already gave a perfect answer. In the test set the darkening was applied to the infected leaves instead, so that learned rule now points precisely the wrong way.
• AUROC below 0.5. AUROC is the probability that the model scores a randomly chosen infected leaf above a randomly chosen healthy one. 0.5 is a coin flip. Below 0.5 means it reliably ranks them the wrong way round, which is not an absence of information, it is real information being applied backwards. (Inverting every prediction would score above 0.5.) It arises exactly when a cue the model learned is anti-correlated with the label in the new data, which is what we engineered here.
• Catching it beforehand. Nothing in the loss curves could have revealed it, and that is the uncomfortable part. In a normal project the training and test sets come from the same batch of photos, so they share the same bias and both look fine. The failure only surfaced because our test set came from a different regime. The general defences: build validation sets that genuinely differ in conditions (other farms, other days, other cameras) instead of randomly splitting one batch; record metadata such as time of day, device and operator, and check whether it predicts the label; probe what the model responds to, for instance by occluding parts of the image or testing on deliberately manipulated versions; and look at the images and how they were acquired, not only at the metrics.
• Mitigations. Image augmentation (step 8). Standardising the acquisition protocol so lighting cannot vary with class. Per-image normalisation, so absolute brightness carries no information at all. Deliberately balancing the confounder across classes at collection time. Holding out a validation set from a different session. Best of all, fixing the data collection rather than patching it afterwards.

• How augmentation helps. By randomly re-rolling brightness, contrast and mirroring on every pass, those properties stop predicting the label: the same healthy leaf is sometimes dark and sometimes bright. The shortcut becomes useless, so the only way left to reduce the loss is to rely on cues that survive the randomisation, the lesions and colour patterns of the leaf itself. As a bonus it enlarges the effective variety of the training set, which reduces plain memorisation too.
• Where the improvement came from. Not from new information, from destroying misleading information. We injected our own knowledge, namely "brightness and mirroring must not matter for this task", as a constraint on what the model is permitted to rely on. That is prior biological knowledge supplied through the data rather than through equations, which is a nice echo of the mechanistic-versus-data-driven contrast from step 0.
• Is it a complete fix? No. Compare it against the step 4 model: augmentation recovers a great deal, but it usually still falls short of the model trained on clean data. Augmentation makes brightness harder to exploit, but the training set is still biased, the model has less clean signal to work with, and the random perturbations add noise to every image. Repairing the data collection would always have been better than patching around it afterwards.
• Rotation versus recolouring. Rotating a leaf by 180° is safe because an upside-down leaf is still the same leaf with the same disease: the label is unchanged, so you are teaching the model a true fact, that orientation is irrelevant. Recolouring green to brown is a disaster because colour is the symptom, browning, yellowing and mottling are how several of these diseases present. You would be manufacturing images whose label no longer matches their content, teaching the model that a brown leaf can be healthy. The rule of thumb: only augment with transformations that genuinely leave the label untouched.
• The optional investigations.
  • Smaller dataset or different class balance: shrinking the data lowers test performance and widens the train/test gap, because small datasets are memorised faster. Pushing the balance towards extreme imbalance makes accuracy look better while AUROC and minority-class recall get worse, the step 3 lesson all over again.
  • Another species: almost certainly poor without retraining. The model has only ever seen tomato leaf shapes and tomato symptoms, and other species differ in both morphology and disease repertoire. You would measure performance on a labelled sample of the new species first, and most likely fine-tune on it rather than assume transfer.
  • Other metrics: precision, recall and F1; balanced accuracy; Cohen's kappa; Matthews correlation coefficient; the area under the precision-recall curve, which is more informative than ROC AUC under strong class imbalance; recall broken down per disease; and calibration measures if you intend to treat the scores as probabilities.