In this digital age, you can hardly glance at a tech blog without seeing a new article touting “big data” − and with good reason. With so much data at our fingertips, we are able to make more accurate predictions and discover new insights by examining data in ways that had never been possible before, with the potential to change how business and organizations fundamentally work.
However, a paper titled “Google Flu Trends Still Appears Sick” was published in the Social Science Research Network recently. This paper outlines the shortcomings of the Google Flu Trends algorithms, which is one of the most touted case studies on big data analytics, calling the results “discouraging.”
I also discovered quite a few articles and blog posts commenting on the reliability of Google Flu Trends, and in a more general sense, questioning the boundless hype surrounding big data analytics. While the issue of transparency in methods and repeatability are certainly concerns, I was left with a more basic question.
“How can we trust any of these so-called ’answers’ derived from big data analysis? What if the correlations we rely upon to make decisions are simply coincidences?” I thought. So I turned to Dave Cieslak (pictured), our Senior Data Scientist, demanding answers.
“Think about it,” he said in response, “if you start looking at large enough sets of data, statistically you’re going to start seeing lots of correlations, if only by chance. You can put data through an algorithm, but the output doesn’t tell you the whole story. You need someone to look at it with a discerning eye and actually translate it into something you can use.”
And that’s when I realized that I needed to refine my thinking. It’s not just about the data. No, it’s when smart, innovative data scientists carefully study the results, and come to conclusions based on their knowledge and experience with data analytics.
It’s like discovering ancient hieroglyphics on a piece of papyrus; even after de-coding it with the Rosetta Stone, you still need an expert to help you understand the context of the words and analyze the deeper meaning. A good data scientist not only helps you decipher your data, but help you get the most from the answers it provides.
That’s why it’s important to work with experienced data scientists you can trust. They can focus on finding the answers that are important to you and that will have the biggest impact on your business practices. After all, it’s not just the data; it’s having those experts in your corner, helping you make the most of it.