The Emergence of Big Data Issues for Agriculture

Published on: 17:46PM Sep 09, 2015


What is 'Big Data' anyway?  According to a 2013 article in the New York Times, it is a phrase coined by a computer scientist at the California-based company Silicon Graphics in the late 1990’s named John Mashey.  There is no consensus as to its definition, but most of its uses seem to refer to a volume of data so large and/or complex that it cannot be easily handled by conventional data management systems.


The availability of massive amounts of data on farm level crop production combined with weather data is relatively new due to the recent widespread adoption of precision agriculture technology.  According to USDA’s Agricultural Resource Management Survey (ARMS), in 1997, only 17 percent of corn acres were cultivated using precision agriculture equipment.  In 2010, the most recent year available for corn production practices, 72 percent of corn acres were planted with precision technology. However, large datasets have been around the U.S. food and agriculture system for many decades.


For example, economists interested in analyzing food consumption patterns and food marketing practices in the United States have received access to scanner data from major grocery store chains since the 1980’s.  Even a single regional grocery store chain can record tens of million separate transactions over a relatively short period like a few months.  Interestingly, one of the businesses involved in collecting and disseminating such data is the A.C. Nielsen Company, better known over the years for collecting information about consumers’ television viewing habits.


The U.S. Department of Agriculture also collects massive amounts of data for programs that it operates, but until fairly recently has not been focused on trying to organize datasets across agencies in a rational way.  For example, it would make sense for a farmer who provides common information for several programs, such as acres planted to various crops in a given year for both the Title I commodity programs and the federal crop insurance program, to be able to report that information to a single agency and then have it transmitted to the other agencies who have need of it as well.


Whether it is data from yield monitors collected and analyzed by John Deere, Monsanto’s Climate Corp. or another big agribusiness company, or information collected by a USDA agency for the purposes of managing a program or surveying the farming public on how their farms are operated, farmers have legitimate concerns as to how the information they provide will be utilized.


Few farmers involved in producing crops over the last few decades will forget the sinking feeling they experienced when the Environmental Working Group (EWG) was successful in obtaining information on farm program payments made to individual farmers under a Freedom of Information Act (FOIA) request back in 2004, and released that information in the form of a searchable database on their website.  While American farmers are proud of what they do, no one likes for their personal financial information to be put on public display, available to be dissected by journalists and/or activists.  As a result, Congress included a provision (Section 1609(b)) in the 2008 farm bill limiting further access to detailed program participation information to outside groups, at the insistence of U.S. farm and commodity groups.  Farmers also don’t want to see their program participation information end up in the hands of other federal agencies, such as the Environmental Protection Agency, to be potentially used in adversarial enforcement actions.


Farmers  have been equally wary about what other parties might do with the information they provide to big agribusinesses like DuPont and Monsanto to enable them to get advice on how to engage in ‘prescriptive agriculture’.  Such parties might include commodity traders or rival farmers.  Those farmers brought their concerns about vaguely-worded, open-ended contracts that failed to address issues of data ownership with these companies to their farm and commodity group state and national leadership.  These communications prompted groups like the American Farm Bureau Federation and National Corn Growers Association to open discussions with the ‘big ag data’ companies in the spring of 2014 over data security and contractual standards.  In November 2014, those groups and the key companies reached an agreement which established principles on data privacy and security with respect to arrangements between farmers and agricultural technology providers.


In later blogs, I plan to explore related issues, such as the potential uses of unmanned aerial vehicles (UAV’s) or drones on farms, and the technical skills that farming now requires of its participants.  It’s clearer now more than ever that the 21st century has indeed arrived on our nation’s farms.