7 tips for keeping Big Data in perspective
Big Data For News Publishers | 29 September 2013
In a previous post, Big Data ambitions were compared to the Gold Rush, when those who provided tools to dig the gold were the ones who earned money in the first place.
This reminds me of the 1990s, when many companies invested in huge customer relationship management (CRM) systems that fell under the umbrella term “database marketing.” These systems often were driven by information and communication technology (ICT) or operational departments, and stressed “database management” over “marketing.”
At that time, information was mainly collected via surveys (declared behaviour) and did not give the business people a great deal of insight. It was difficult to effectively track proven behaviour and, therefore, create uplifts for marketers. So they lost interest.
Since the digital revolution, publishers are in a better position than ever before to collect plenty of information about consumers and their habits, needs, and wants. This daily, interactive contact with large audiences generates “big” data. So “big” systems are needed to capture this proven behaviour.
For many companies, Big Data equals Hadoop, the system that collects all that big data, regardless of why they need it and how they plan to use it. When gathered this way, however, Big Data becomes simply “Big Useless Data” that creates noise and obscures the signal in our data. The noise is increasing faster than the signal.
Then again, just like in the ’90s, right-brained editorial and business staff will quickly lose interest in it.
No wonder, then, that a new analyst report indicates that enterprises are deriving far less value from Big Data than they expect, or even than they invest.
Dramatically less, according to preliminary findings from Wikibon research, which found 46% of Big Data practitioners were only partially successful with their projects. They hear it’s a big deal and throw money at it without really understanding what they’re hoping to achieve. Two percent even had to write off their investments as complete failures.
Just like the CRM-bubble in the ’90s….
So, how to avoid this booby trap and make sure your big data investments pay off?
In the ’90s, marketers were the right-brained, creative, and intuitive “Mad Men.” If we want our Big Data investments to pay off, they have to become “Math Men.”
Data should be their new weapon of choice, but modern marketers today look for ways to use data to become more efficient in serving customers, boost figures, and, last but not least, improve customer experience.
To illustrate this: Last month, 20 top Belgian entrepreneurs, ad agency managers, and politicians spent a week traveling around Silicon Valley to immerse themselves in the fascinating stories of the American West Coast. They visited several companies and startups and came back with some good examples of where Big Data was undoubtedly the fuel for the business.
For example, Evernote, a free app to create notes and record voice memos, is convincing consumers to switch to their paid version by presenting commercial messages in the appropriate context. Kind of a selective paywall, isn’t it?
Nike’s Fuelband is another example of a Big Data tool. Users of this digital bracelet provide Nike with tons of free personal info. Nike knows where you are, how far you walk (or run), how often you work out, and what your heart rate is. Nike could have a new pair of trainers delivered to your doorstep at exactly the right time.
Compare those capabilities to the news media industry. We could derive from our data the ultimate proactive service a media brand could offer its audience.
All Big Data success stories have one thing in common: Data is not a goal; it’s just a means to improve both companies’ efficiency and customer experience.
Relying solely on gut-based decision-making is foolish in this digital age. The sensible answer for most media companies is a balance between data analytics and human judgment; between quantitative and qualitative; between creative above the line branding and responsive direct marketing; between science and art; between “push” and “pull”; between “surprising content” you didn’t ask for but found inspiring and “news on demand,” tailored and individualised upon request, just like Amazon does.
In the spirit of a more balanced approach, to avoid data-drowning and over-investment, and to support the business and editorial staff with a pragmatic approach to embrace Big Data, I collected seven tips to keep your data in perspective of the business partners:
- Visualise the data to make it tangible. Left-brained “mad men” remember pictures best. So don’t bombard them with spreadsheets, where they have to drill down themselves for results. Use vivid and accessible data visualisation tools — graphs, charts, infographics, etc. — to give them insights. It should be fun to look into dashboards (note: data visualisation is a science and an art unto itself).
Visualisation is the most effective way for us humans to see the patterns in data. But be aware: It can not only illuminate but also obfuscate or distract, too. So don’t become myopically glued to your dashboard; look out the windshield, too, to avoid accidents.
- Don’t be data-driven; be customer-driven. Data-driven marketing is good. But the purpose is to win customers (and keep them by making sure they have a great experience).
Data is the means and the customer is the end. It helps to remember that before connecting yet another source of data to your “single customer view,” you should stop and ask, “How is this good for my customers?”
- Data is objective, but data collection and interpretation are subjective. “Data ends arguments.” But with so much data swirling around, it’s always easy to find data to support nearly any side of an argument.
But data differs in its accuracy and its relevance. So a better motto might be: The most accurate and relevant data ends arguments. (But in truth, it’s a balanced decision that ends arguments.)
- A man with a watch knows what time it is. A man with two watches is never sure. Just try to get audience metrics from two separate Web analytics packages to know this truth. Different tools will measure the same phenomenon differently. Understanding why they’re different might lead to valuable insights, but there’s a diminishing return to chasing every minute discrepancy.
In many cases, you don’t need perfectly accurate data, but simply sufficiently accurate data to make good decisions. Tip: There are so many hypotheses to test, so many data sets to mine, but only a relatively constant amount of objective truth, often easily detectable in “small data.”
- Publishers are storytellers; use your data to make the narrative more compelling. Data can be used to make stories more compelling. But, since we can subjectively choose how to interpret data, we can invent almost any narrative we want around it.
It helps to keep perspective.
Any story presented around data is never the only story that could be told about it. Both in editorial as in commercial messages, any set of data supports an infinite number of narratives. (Luckily!)
- Correlation is not causation. Every decent data scientist will tell you this. But marketers and editors are usually after causation, not only correlation. So when data shows a correlation that might reveal a cause, you will have to run a controlled experiment to help them out. Keep all other variables constant (as much as feasible in practice) and test the alternatives to prove or disprove the hypothesis.
- Test-data is the most powerful data. Google runs more than 10,000 data experiments and tests every year to improve its products, efficiency, and services. The goal of those tests is not data, per se; the goal is to make your data actionable.
Before you’re modeling your data into actionable data, you need tests to verify if your model generates the uplifts you’re seeking.
First, test to seek data patterns, insights, ideas, and discoveries. And then scale it into models. That’s why test-data is the most powerful kind of data, and that’s why big testing will be bigger than Big Data itself.
So, what about you ? How do you make sure the data is leveraged in your (media) company? Did you run tests in the past to explore uplifts in your business? Data-driven tests (upon big or small data) you are willing to share with other publishers around the globe? Then raise your hand and drop me an e-mail at firstname.lastname@example.org.