One of the things we preach in Lean Analytics is that entrepreneurs should avoid vanity metrics—numbers that make you feel good, but ultimately, don’t change your behavior. Vanity metrics (such as “total visitors”) tend to go “up and to the right” but don’t tell you much about how you’re doing.
Many people find solace in graphs that go up and to the right. The metric “Total number of people who have visited my restaurant” will always increase; but on its own it doesn’t tell you anything about the health of the business. It’s just head-in-the-sand comforting.
A good metric is often a comparative rate or ratio. Consider what happens when you put the word “per” before or after a metric. “Restaurant visitors per day” is vastly more meaningful. Time is the universal denominator, since the universe moves inexorably forwards. But there are plenty of other good ratios. For example, “revenue per restaurant visitor” matters a lot, since it tells you what each diner contributes.
What’s an active user, anyway?
For many businesses, the go-to metric revolves around “active users.” In a mobile app or software-as-a-service business, only some percentage of people are actively engaged. In a media site, only some percentage uses the site each day. And in a loyalty-focused e-commerce company, only some buyers are active.
This is true of more traditional businesses, too. Only a percentage of citizens are actively engaged in local government; only a certain number of employees are using the Intranet; only a percentage of coffee shop patrons return daily.
Unfortunately, saying “measure active users” begs the question: What’s active, anyway?
To figure this out, you need to look at your business model. Not your business plan, which is a hypothetical projection of how you’ll fare, but your business model. If you’re running a lemonade stand, your business model likely has a few key assumptions:
- The cost of lemonade;
- The amount of foot traffic past your stand;
- The percent of passers-by who will buy from you;
- The price they are willing to pay.
Our Lean lemonade stand would then set about testing and improving each metric, running experiments to find the best street corner, or determine the optimal price.
Lemonade stands are wonderfully simple, so your business may have many other assumptions, but it is essential that you quantify them and state them so you can then focus on improving them, one by one, until your business model and reality align. In a restaurant, for example, these assumptions might be, “we will have at least 50 diners a day” or “diners will spend on average $20 a meal.”
The activity you want changes
We believe most new companies and products go through five distinct stages of growth:
- Empathy, where you figure out what problem you’re solving and what solution people want;
- Stickiness, where you measure how many people adopt your solution rather than trying it and leaving;
- Virality, where you maximize word-of-mouth and references;
- Revenue, where you pour some part of your revenues back into paid acquisition or advertising;
- Scale, where you grow the business through automation, delegation, and process.
The kind of activity that matters most to you changes at each of these stages. In the empathy stage, it might be the number of people interviewed who try to buy your product right away, or keep pestering you on your mailing list about it before you launch a beta. Later, in stickiness, it might be the number of people who return within a particular timeframe. And in the virality phase it might be the number of people who are actively telling others.
The point is that an active user is someone who does the thing your business needs most with the frequency you need them to. One company Ben and I worked with had defined active users as people who logged in daily, and their numbers were dismal. But when they called users, they found that many of them were using the product in a weekly manner, and not in the way the founders had intended. This allowed them to modify the product, but also to redefine “active” as “using the product once a week.”
Every metric is interrelated
If you’re trying to maximize a certain kind of activity, other assumptions will change, too. In our lemonade stand, there is probably a relationship between the price of lemonade and the percent of passers-by who buy some. Charge $10 a glass and you won’t sell any; charge $0.01 and you might sell more, but will also lose money. But in the middle are a variety of price options. Consider two budding entrepreneurs:
- Jane runs an acquisition-focused stand. She wants to make one, and only one, sale for as much money as possible—she’s gouging friends and relatives. For her, “active customers” might be “people who pay $5 for a tiny glass.” Her goal is to take as much money as she can knowing a customer won’t return. She might charge as much as possible, and even sell decorated cups or charge extra for ice.
- John runs a loyalty-focused stand. He wants to sell lemonade every day to people walking home from a bus stop. His goal at this point might be to spread the word about his lemonade, so “active customers” might be “customers who are telling their friends.” He might charge less money to encourage maximum virality.
Most smart entrepreneurs have moved beyond vanity metrics at this point. But they get bogged down in defining what “active” looks like. Generally, the way to figure this out is to look at what stage your company is at, and what your business model looks like, and to decide what assumption has the biggest impact on your organization right now.
On August 6, along with Eric Ries, we’ll be talking about Lean Analytics 201, and diving in beyond the basics to discuss these kinds of issues. We hope you’ll join us.