Sifting Through All The Big Hype About Big Data

Posted by Taylor Haney on Fri, Apr 25, 2014

As we start to wrap up our Data & Analytics theme for the month we have a post from Niren Sirohi, Vice President of Predictive Analytics at iKnowtion. Niren provides some very helpful rules to keep in mind when trying to assess the world of big data and what solution will be best for you.

Niren SirohiNiren Sirohi, Ph.D. is Vice President of Predictive Analytics at Boston-based analytics firm, iKnowtion. Contact him at @sirohin

Today, it is hard to escape all the chatter about Big Data. Every time I read an article or a news story around Big Data, I can imagine the frustration every marketer must face as they try to decide what the best course of action is in this brave new world that is rife with contrarian viewpoints. Hence, I thought it would be valuable to take a critical look and distill some fundamental rules that should govern marketers' actions in the big world of Big Data.

Expensive technology solutions are not the panacea: Much of the Big Data conversations today originate from technology vendors and involve terms like the cloud, Hadoop, Map Reduce, Analytic appliances, Massively Parallel computing, etc. These conversations have little if anything to do with the business case/ROI of Big Data. Although all of these concepts are important, just like in the CRM era, focusing only on technology solutions and trying to build a 360-degree view of the customer will lead to frustration. Instead, one should start with identifying the value of the various sources of new data through small focused efforts. These efforts can be implemented at the grassroots level without investing significant amounts in technology.

Invest in the right skills before technology: More important than technology is having the right skills. Three key skills are required. First is the ability to frame and ask the right business questions of the data with a clear line of sight as to how the insights will be used. Second is the ability to use disparate open source software to integrate structured and unstructured data. Third is the ability to bring the right statistical tools to bear on the data to perform predictive analytics and generate forward looking insights.

Data Poor, Insight Rich is much better than Data Rich, Insight Poor: Never before has this been as true as it is now. The risk of data overload without commensurate insights is at its peak. The reality is that most organizations have barely leveraged the information they iKnowtion blogalready had, even before the world of Big Data. Those who have generated insights have barely scratched the surface of being able to implement and act on those insights at the frontline, where they really matter. Generating meaningful insights and acting on them should be the first order of business, before new sources of data are exploited.

Be prepared to deal with new challenges: In the small data world, a lot of emphasis was placed on having complete data without any holes at the individual level. In the new world of Big Data, be prepared to deal with vast holes of information at the individual level; new analytic techniques are required to deal with this information effectively. Also, due to privacy reasons, several streams of Big Data (e.g. social media data) will never be trackable to a specific individual, limiting the use of this information for individual targeting purposes.

Real-time is sexy, but usually not necessary: Ask yourself the following questions before falling prey to the real-time hype. Can you collect data in real-time? Can you analyze it in real-time? Can you make decisions in real-time? Can you act on those decisions in real-time? How much difference in customer value is created if you do all of the above in real-time? Only when the answers to the first 4 questions is Yes and the answer to the 5th question is significantly larger than the cost of making things happen in real-time, should you consider real-time as a viable option.

If you would like to learn more about how you can turn Big Data Analytics into a competitive advantage, feel free contact me directly at