Computer vision
How Syngenta taught its machines to see crops, weeds and pests
Farming has always depended on the human eye. Now a new generation of computer vision tools is changing what it means to look at a crop, and how much can be learned from a single image.
Rob Lind, Syngenta Fellow in Computer Vision and Artificial Intelligence, had three hobbies growing up—insects, computing and photography—and found it difficult to choose one to pursue at university. He ultimately chose biology, took a doctorate in the neurophysiology of insecticides, and joined the company that would become Syngenta to head up insecticide screening.
Rob Lind, Syngenta Fellow, Computer Vision & AI.
Rob Lind, Syngenta Fellow, Computer Vision & AI.
The job came down to numbers. Live insects in one column, dead ones in another, the difference between them standing in for whether a compound had worked. "Nobody really enjoys counting lots of things," he said. "I've had to do it as a job, and it isn't much fun."
The tedium was the least of it. The figures could not quite be trusted. Hand two people the same pot of insects and they will come back with different totals. Ask one person to assess the same trial on a fresh Monday morning and again on a tired Friday afternoon, and they may produce different results.
The seeds of a solution to this problem came during his doctorate at the University of Bath, where another PhD student, Iain Couzin, was trying to track the movements of an ant colony, hundreds of insects at a time. Realizing that tracking by eye would be impossible, they set up cameras and made one of the earliest attempts to teach a computer to track moving objects from a camera feed. This was the late 1990s, before digital cameras were common and long before cloud computing existed. "I was really blown away by what he did," Rob said.
When his own lab work began, Rob hired Iain as a contractor and set about doing the same thing for insecticide screens. Today, Rob leads computer vision at Syngenta, building the systems that teach machines to see what is growing in a field, and what is threatening it.
The trouble with the human eye
"Computer vision is about teaching computers to see the world and understand what they're looking at," Rob said, whether that is a crop, a weed, a fruit or a pest. Its appeal to a research company lies less in sharper sight than in consistency, and in the ability to put a firm number on things the eye can only estimate: how red an apple is, how far a caterpillar has crawled, how many plants stand in a row.
For most of history, that judgment belonged to the human eye. Until around the year 2000, by his account, the biological assessments were made almost entirely by people studying the plants themselves.
When agriculture companies are weighing two candidate compounds for a place in its development pipeline and the gap between them is narrow, subjective, non-quantitative human assessment, as he puts it, "get lost in the noise." A model does not.
What makes the machine version possible is a vast store of pictures. Agri-tech leaders such as Syngenta hold around half a billion agricultural images, most of them taken in their trialing fields, to help models to recognize what matters. Compared with the visual libraries that driverless-car companies have amassed, Rob notes, this figure is fairly modest. Set against farming, it is an enormous amount.
Tools in the field
For farmers, the technology usually shows up as something simpler. In Brazil, growers hang pheromone traps that draw in crop-damaging moths, sometimes several hundred at a time, and the decision of whether to spray a field can hinge on how many there are. Counting them by eye is slow and unreliable, so Syngenta built a phone app that does it from a single photograph.
This year the team released a similar tool for corn, which tallies the kernels on an ear from an image and feeds into a yield estimate. Many of these tools are small enough to run on the phone itself rather than on a distant server. "You don't need to transmit gigabytes of imagery back to the cloud," Rob said.
The hardest cases tend to be the ones that would defeat a person too. Plants growing close together begin to overlap and hide one another, and teaching software to pull them apart is genuinely difficult. "If I can do it, the neural network should be able to do it too," Rob said, and the reverse holds just as well: when a human cannot make sense of an image, the machine usually cannot either.
Calculating metrics of virtual sprays from drone images.
Calculating metrics of virtual sprays from drone images.
Some jobs sit right on the line. Overlapping cereal heads, he says, the networks handle reasonably well. The tangled leaves of wheat plants are close to impossible.
The models themselves have shifted. The turning point came in 2012, when a system called AlexNet, built by a team at the University of Toronto, won a research contest in which algorithms were shown millions of labeled photographs and asked to name everyday objects. It won hands down. "That's when people sat up and thought, this is big," Rob said.
AlexNet established convolutional neural networks (CNNs) that are compact enough to live on a phone. The newer arrivals are larger and more capable: vision language models, the same broad lineage as today's chatbots, which can take in a picture and a written question and work across both at once. Syngenta has recently begun experimenting with these transformer models, through a collaboration with Google.
Machine that moves and sees
About a third of Syngenta's field trials in crop protection development are now flown by drone, and the machines are good at gathering images and useless at understanding them. The problem is power. A drone needs most of what it has just to stay in the air, with nothing to spare for the heavy work of running a vision model. So, the images come down the hard way. The drone lands, someone pulls out the memory card, the pictures are uploaded to the cloud, the models run there, and only then do the results come back. A person has to stand in the middle of the chain.
The robodog is Rob's answer to that bottleneck. It carries cameras and a processor strong enough to run its vision models out in the field, picking out weeds and crop threats as it goes, so it can skip the upload entirely and send back the answer rather than the footage. He calls it a supercomputer on legs, “capturing leaf-level data that helps our trialist make biological assessments and perhaps, one day, help agronomists and farmers catch problems earlier and treat them more precisely.”
The first camera system Rob built, in 1999, to digitize those insecticide screens captured three-tenths of a megapixel through an analog sensor, its signal converted to digital by a special board slotted into a PC. The computing power in a modern smartphone, he points out, would now outperform the lot.
A device that fits in a shirt pocket, bought off a shelf in any city in the world, would today outclass a purpose-built professional rig that once represented the cutting edge of imaging. "Sometimes I have to pinch myself," he said. "Go back thirty years, and I'd never have imagined I'd be running this kind of program." Somewhere in that span, the boy who could not choose among insects, computing and photography found that he no longer had to.
