Are homebrewed + out-of-the-box analytics the data answer?

By Ariane Bernard

INMA

New York, Paris

Connect      

The art (and science) of analytics is measuring things. And the art of contextualising them lies in statistics — understanding what the numbers mean against other numbers that we understand and control.

Cassie Kozyrkov, the chief decision officer at Google (yes, that’s a title) — who has an awesome series on data science, analytics, and statistical concepts on YouTube (sidebar: I really recommend it and it’s very accessible) — says: “Analytics is what helps you ask better questions, whereas statistics is what gets you better answers.”

Do the analytics tools you're using help you answer the right questions?
Do the analytics tools you're using help you answer the right questions?

So look at your current crop of tools, and ask yourself: What are things that you either don’t measure at all or where the statistical understanding of what is observed is lacking.

I’ll give you a couple of ideas that may be relevant for your organisation (because these things aren’t well covered by commercial tools):

  • What is the value of visiting a particular piece of content in a user’s journey (does visiting article A do more than usual in moving our visitor toward their next funnel event)?

  • Is an article a sleeper hit? (Which translates to: “Given the amount of promotion the article has been given, does it manage to get more clicks than you’d expect”)?

What these ideas have in common (and I’m not even scratching the surface of the questions you could throw there) is that answering these questions relies on tying up data that’s going to come from at least two of your publishing systems. In other words, the reason you cannot usually get answers to these questions from commercial tools is that commercial tools live in one dimension (your Web site). And to answer the questions above, you need to cross at least two systems together.

The question, “What is the value of visiting a particular piece of content in a user’s journey?” (ie, does visiting article A do more than usual in moving our visitor toward their next funnel event?), can be answered by bringing up data from your paywall and your CRM. Because, for metered models, the paywall may not have been triggered on a particular article, yet you may find that a given article is represented above average looking back in the sessions of folks who converted over a recent period. You still need data from your regular content analytics tool because you need to have a statistical sense of the chances of any random URL to be present in the session of a converting user.

The question, “Is an article a sleeper hit? (which translates to: “Given the amount of promotion the article has been given, does it manage to get more click than you’d expect?),” can be answered by crossing the data from your social publishing tool (or your CMS if this happens directly from your CMS) and your regular content analytics. In addition, if you can pull ranking information from your CMS about any on-site promotion (like homepage play), this further refines such a number. 

This can get pretty fancy.  

The Telegraph in the UK created a score called STARS (Simple Telegraph Attraction and Retention Score), which tracks articles and scores them by tying up engagement metrics and commercial performance (subscriptions). 

The UK's Telegraph's STARS (Simple Telegraph Attraction and Retention Score) scores articles by engagement and paid subscriptions.
The UK's Telegraph's STARS (Simple Telegraph Attraction and Retention Score) scores articles by engagement and paid subscriptions.

Over at ​​The Times in London, where some similar worries about the lack of context for raw metrics led the company down the path to build its own data stack altogether, Dan Gilbert, then-director of data for News UK (now their SVP), provided background in a blog post for their homebrewed tool that helped augment that contextual understanding of performance.

One place where The Times looked to smooth out comparables was around play (placement) or length of articles. This data doesn’t come from your analytics per say (it would come from your CMS). The Times tools lean on both aspects of the quote from Cassie Kozyrkov — it’s better analytics but also better statistics because it cares deeply about meaningful referential for the analysis.

If you’d like to subscribe to my bi-weekly newsletter, INMA members can do so here. 

About Ariane Bernard

By continuing to browse or by clicking “ACCEPT,” you agree to the storing of cookies on your device to enhance your site experience. To learn more about how we use cookies, please see our privacy policy.
x

I ACCEPT