Sorting Through Big Ag Data Part 2

  • Many are concerned with the privacy issues associated with precision agriculture data. These are real concerns.  I have worked with private farm data all my career, but precision maps are a particularly difficult kind of data to protect as geo-referenced maps can be quickly matched with other geo-located files and identities revealed.  Because of the size of data and for security reasons, I think we will see this data storied securely and rather than sharing data sets, and analysts will come to the data, run analysis, and walk away with results only.
  • Many producers will interact/share data with various machinery companies, input suppliers, consultants and others. Some will want to lock you into their system, and many producers are faced with the challenge of moving information such as a field boundary map or fertility prescription to various users.  Producers need a safe place to store this data, and seamless means to move this information to fertilizer applicators, USDA/FSA.  I am proud that Mississippi State University is a founding member of the Ag Data Coalition that involves farm organizations, agribusiness firms, and universities committed to helping producers control and utilize their data.  I encourage you to check it out at http://agdatacoalition.org/
  • Getting advice from a black box. It seems that almost every input supplier now has an app or software that is designed to help farmers make decisions.  In some cases, these are going to be really useful farm management tools.  There are some that impress me a lot. I also believe that some are way to entice farmers to give data to service provider or to make it difficult to leave a service provider.  I am also struck by the similarity of the sales pitches made by some big data based decision aids and that of the market advisory services that have a great sales pitch but when evaluated, have no better results than flipping a coin.
  • I just helped organize a conference on big ag data at the Agricultural and Applied Economics meetings. We had big name speakers from many disciplines speak.  In the end, there are limits on what one can learn from unstructured ag data that may not include some really important variables.  Using machine learning to find correlations among millions of records can lead to new discovery, but it can also find spurious correlations that may lead to bad decisions.  We will still need well designed statistical studies to answer some big questions about causation.
  • Big data begs several policy questions such as rural broadband access, funding for research on the topic, and the role of USDA data collection. I once worked at USDA and saw the hard work that goes into collecting agricultural data that is unbiased and statistically sound.  However, over my entire career I have watched as hundreds of ‘experts’ make a living claiming to know better than the USDA reports.   What you can’t find is a multitude of studies assessing the accuracy of these experts like one can find for the USDA reports.  I think we are beginning to see a new wave of ‘we know better than USDA’ business models that charge a fee and have not been verified to really be better.  Let the buyer beware, and let’s remember why USDA was asked to do these reports – to insure everybody in the ag sector had equal access to market information.