The big data dairy challenge

The dairy sector is showing how machine learning and artificial intelligence systems can convert raw data into insight

Reading Time: 5 minutes

Published: May 18, 2018

The big data dairy challenge

Dairy farm software and equipment is increasingly turning to the power of machine learning and big data sets to bring new tools to farmers.

Large, established companies and startups are bringing their scale and creativity to bear on the challenge and the results are opportunities for on-farm management farmers haven’t seen before.

Why it matters: The dairy sector has led in automation with robotics, sensors and with a long history of managing large data sets. Dairy organizations and companies are now dabbling with computers that can make decisions on how data is used.

Read Also

Dr. Greg Aitken, a large animal veterinarian at Peterborough Veterinary Services, demonstrates the ease with which Solvet's Lidoband, a lidocaine-infused latex band, can be administered on lambs for tail docking or castration.

Ontario Sheep Field Day returns with focus on welfare tech and productivity gains

Veterinary demos, including Solvet’s pain-reducing Lidoband castration, highlight innovation for commercial sheep producers at Ontario Sheep Field Day

CanWest DHI started out testing fat in milk decades ago. Then it moved to protein, followed by other quality and health indicators such as somatic cell count and ketosis. Now, CanWest DHI general manager, Neil Petreny says the organization collects 1,100 data points per milk sample.

Big data sets collected over many years can help monitor current production and health trends in the industry, but also will create the opportunity to look back and evaluate milk production historically.

Dairy farmers have been the big data leaders in agriculture for decades, with the collection of animal genetic evaluation information through the Canadian Dairy Network (CDN), and milk quality information through the dairy herd management organizations and their on-farm testing.

“Data now is starting to grow exponentially,” said Petreny, whose organization is in the process of creating a partnership with its Quebec and eastern Canadian counterpart Valacta and CDN that will create one brand for dairy data services in Canada.

Robotic milking systems collect 150 to 200 data points on each animal per day. photo: John Greig

Petreny says milking robots can collect 150 to 200 data points on an animal each day and there are hundreds of sensors available or being researched for dairy farms — all producing more and more data.

In the recent past, companies and organizations have been able to write algorithms that manage data collected. Companies such as sensor and robotic leaders DeLaval and Lely have systems that recommend actions instead of feeding farmers raw data. But the ability to write algorithms fast enough to manage increasing volumes of data are ending. That’s where machine learning or artificial intelligence can assist because a machine can manage the interpretation of data based on what it already knows.

Sensor startup

Bethany Deshpande leads one of the most-hyped startup companies in the dairy data space. She had just finished her Phd, when her father, Satish Deshpande, an inventor, figured out how to use sensors to detect milk fat and somatic cells in milk.

She’s now the president of SomaDetect, a company with an in-line sensor that measures milk fat, protein and somatic cell content, along with progesterone for pregnancy and heat detection and ketones for the detection of ketosis.

SomaDetect is able to measure those items in milk, without any reagents that have to be purchased to do the measuring or any milk that needs to be thrown away.

However, on top of the sensor innovations, the power of SomaDetect is in machine learning. The data is uploaded to the internet and processed by systems that have the ability to learn with the data.

Bethany Deshpande. photo: John Greig

“What we do at SomaDetect would not have been possible even five years ago,” said Deshpande at the Canadian Dairy Xpo.

Her company is based in New Brunswick, and Buffalo, New York after the company won 43 North, a startup competition based in Buffalo. It gave them $1 million to develop technology and access to business mentors.

The company is still in the development phase with sensors on farms in New Brunswick and now at Cornell University. It is looking for innovative farmers in Ontario to try out the technology. It is easy to install, with a box hooked into the milk line, powered over the Ethernet cord it uses to move data. One SomaDetect unit is needed for each milking unit, so the system becomes more expensive for parlours versus robotic systems. It costs $2,000 per sensor and $4 per cow per month, although they are waiving the cost for data and analytics for the first year, and charging half price for the second year. Other in-line analysis technologies for the milk constituents tested by SomaDetect require reagents and testing strips that have to be purchased and reloaded. SomaDetect’s technology measures the components as the milk flows with light.

“What SomaDetect has done and what we do really well is applying these in the dairy industry. It allows us to pull a ton of information and achieve higher levels of accuracy than people have seen before,” said Deshpande.

SomaDetect’s sensors installed in a milking parlour at Cornell University. photo: Courtesy SomaDetect

Managing all that data

One of the largest challenges for farmers in this explosion of monitoring options is the challenge of managing all the different options, especially when it comes to software. Farmers don’t want 20 different apps on their phones and computers in order to monitor their herd. The challenge for new companies is to figure out how to get their software integrated with the systems of with larger dairy companies.

It’s not easy, says Deshpande.

Large companies are taking a run at being the home for all that dairy data.

Purina recently launched Dairy Enteligen, a complex software system that supplies their nutrition advisors with up-to-the-minute information on feed ingredient availability and quality.

It also integrates farm level data to help farmers make better decisions.

“Data analytics are not integrated into overall management,” said Mike Jerred, global technology manager, dairy, for Cargill, the owners of Purina in Canada.

The goal is to give farmers the ability to make business decisions.

If a farmer adds 10 more cows, he or she wants to not only make it easy to understand what it means for milk production, but also for the business.

The Enteligen system will also allow farmers to benchmark their operations against other farms.

“This will take us from theoretical to real-time models,” said Jerred.

Data going into the Enteligen system is put through a third-party company so that data from more than 40 different dairy software programs can be standardized for use in the system.

“We want to make sure we are more data driven and move away from more of the gut feel,” he said.

Data and ownership

There remain concerns among farmers about trust in who owns the data.

Petreny said that at CanWest DHI farmers are able to opt out of the use of their data for more than just their own comparisons, without contributing to benchmarking and breed improvement programs.

Jerred calls data ownership “a critical thing.”

The data is owned by farmers, he said, and at any point farmers can remove their data or restrict how it is used and who can see it.

Like all technology that relies on data, the output is only as valuable as the data that enters the system. Garbage in — garbage out. The foundational data has to be solid, said Petreny, or the decisions that are made by the system will be off from the start. That’s why information to prove something such as genomic testing of cattle is so important.

Farmers will have significant decisions to make on which sensor to use and data systems to trust. The only problem they won’t have is enough options.

About the author

John Greig

John Greig

Senior Editor

John Greig is a senior editor with Glacier FarmMedia with responsibility for Technology, Livestock and Ontario. He lives on a farm near Ailsa Craig, Ontario. Contact John at [email protected] or follow him on Twitter @jgreig.

explore

Stories from our other publications