Measuring Impact — Picking the right metrics in Product

Why picking the right metric to track success is so important and, yet, so difficult

Picking a metric is easy. Picking the right metric is a lot harder. With the movement in the product community to ensure we’re building outcome-focused products instead of simply shipping features — the question of picking the right success metric is one that comes up time and time again.

I’ve always loved using data but over my career have made lots of mistakes in how I used data and measured things. Over time, thanks to working with amazing colleagues across product and analytics on different projects, I’ve built a better understand of how different metrics work together and how we can better measure our product’s success.

A model for thinking about metrics

Making a change to your product is similar to throwing a stone into a still lake. Visualise it. Before you throw the stone the water is smooth and untouched. When you throw the stone, you’re going to see a big splash where the stone enters the water. From the epicentre, ripples will spread outwards but with a decreasing size. The size of the ripple next to where the stone landed is much bigger than the ripples further away. In fact, if you look far enough away from where you threw the stone you may not see any ripples at all.

Making changes to your product affects metrics in a similar way. The stone (the changes you’ve made) will generally have a more noticeable impact closer to where the change was made. The more removed a metric is from the change the harder it will be for you see to an impact or movement.

For this article, let’s use an example from an b2c transactional product:

Example Case: We are the Product Team looking after the funnel for an accommodation rental company. We’ve see there is large drop off in the funnel…