5 All-Too-Common Data Analysis Mistakes

Posted by Taylor Haney on Wed, Aug 21, 2013

It's still all things data this month. Continuing on with our guest blog series we have a must-read post from Tom Shapiro, Founder of Digital Marketing NOW. Tom addresses some common data analysis mistakes and how to avoid them. Interested is guest blogging, e-mail taylor [at] mitx [dot] org for more information.

tom shapiro 01Tom Shapiro (@TomShapiro) is founder of Digital Marketing NOW, a strategy, design and marketing agency offering branding, marketing strategy, web development and design, SEO, paid search, display advertising, analytics and more. Throughout his career, Shapiro has worked with a range of Fortune 500 clients, including P&G, HP, Intel, AT&T, and Kraft Foods. Hear Shapiro speak at the upcoming FutureM conference on October 18 about hot new digital marketing trends.

We now live in a world of big data. Ninety-percent of all the world’s data was created in the past two years, according to IBM. And the data deluge continues. The amount of data created in 2012 reached 2.8 zettabytes (Err, I cannot even imagine how large a zettabyte must be!), and according to IDC this number is expected to double by 2015.

With so much data, you’d think that data analysis would be a marketing wonderland. However, the reality is quite different. We still live in a world where flawed analysis is the norm at too many organizations. This is a wasted opportunity, given all the juicy bits of data surrounding every marketing campaign, initiative and program now being implemented.

Here are five examples of all-too-common data analysis mistakes, and ways you can avoid them:

1. Focusing on Incomplete Data Fragments

Looking at surface-level data can be misleading. For example, examining Page View trends may not really help you much. It’s possible that your best site visitors are not those that look at the most pages. High Page View counts can sometimes reveal that something’s actually wrong with your site, such as complex navigation that drives extra, needless clicks.

However, if you cross analyze your data, you’ll have much more usable data to drive your decisions. Instead of looking at just Page Views in a silo, analyze the Page Views of new visitors vs. repeat visitors, or Page Views sourced from SEO vs. display ads, or Page Views for those who converted on-site vs. those who left without converting. It’s the cross analysis of data that provides you with rich, actionable context.

2. Focusing on Meaningless Numbers

I still hear marketers talking about (oftentimes) meaningless metrics such as Time On Site. Yet when pressed on the correlation between Time On Site and lead generation or revenue, the answer is typically hollow.

Instead of collecting as many data points as possible, focus strategically instead on data that's actionable. Are you focusing your time and attention on metrics that lead to your stated goals, or instead on metrics that merely fill presentation slides?

3. Confusing Correlation with Causation

Just because two things happen concurrently does not mean that one caused the other. This is a common data analysis misstep that can lead to wildly inaccurate conclusions.

Let’s say that your Facebook Likes increase greatly at the same time as your revenue. It would be natural to assume that the Likes were causing the increased revenue, and you may double down on the same types of Facebook marketing initiatives. It’s quite possible, though, that on deeper analysis the Facebook Likes were caused by an isolated incident or even spam. To identify your most effective marketing, you need to dig deep to uncover true causation or you risk making decisions based on random events.

4. Making Decisions Based on Statistically Invalid Data Sets

Being aggressive in marketing is great, except when it isn’t. Often with highly aggressive, fast-growth, tech companies, the inclination is to move fast in making changes to marketing programs. However, making decisions based on relatively small data sets is risky and sometimes very misleading. Better marketing decisions could be made by being more patient with data collection and capturing more statistically sound data sets, leading to more significant, more sustainable and longer-term growth.

5. Forgetting Geography

For whatever reason, geography is often a forgotten element in data analysis. If you do not sell internationally, it doesn’t make sense to include international data in your analyses. If you sell internationally, you probably have high priority markets that should be analyzed separately from other markets. If you sell locally, looking at national data may be misleading. And on the flip side, if you sell nationally, looking at your local markets may lead to completely new geo-based insights.

Yet many companies look at their analytics without an effective level of geographic filtering. Make sure your data matches your analysis needs, or you could be analyzing skewed data.