What are Vanity Metrics? And How to Avoid Them.

Not all metrics are created equal. It takes time, effort, and a strong understanding of your business to identify the best metrics to track for your situation. Metrics fall into two categories for most companies: vanity metrics, or actionable metrics (Key Performance Indicators). Today, we are discussing what exactly makes a vanity metric so we can either avoid tracking something of no value, or better yet, turn that metric into a KPI.  To start, let's take a look at the definition of vanity.

Vanity:

1. excessive pride in or admiration of one's own appearance or achievements.

2. the quality of being worthless or futile.

For startup founders, both of these definitions can apply when you are faced with one of the most dangerous issues for early companies, tracking (and often hyping) the wrong things. Let's break this definition down, from the perspective of a startup founder.

Vanity Metric:

1. excessive pride in or admiration of a company's baseline metrics.

2. the quality of the metric is worthless or futile.

Alright, what do we mean by this? Let's start with part one of the definition.

Excessive Pride & Admiration

You've been working on/living your company for weeks, months, maybe years. You are enamored by it, cry over it, fight for it. So you launch your first iteration of your product. Maybe you install something like Google Analytics to track your impending greatness. You fire up your analytics tool... and you have 1,000 pageviews and 10 daily active users in your first month. You have made it. Someone looked at a page on your website 1,000 times and people are coming to your site. This is real. You keep up the hustle, and the next month you have 1,200 pageviews and 8 daily active users. Well 1,000 to 1,200 sounds great! And 10 to 8 isn't so bad. So we hype up the pageview growth of 1,000 to 1,200 to friends, family, and (yikes) maybe even early investors. What's wrong with that?

Don't make the puppy sad... find your KPIs

Well, what happened is your pride & admiration of your own company blinded you to some troubling things that need immediate attention. In this scenario your daily active users dropped 20% since the previous month. Not good. And while that was happening you had a 20% increase in pageviews. Which could mean a lot of things. Could be your site is difficult to navigate and people are clicking around a lot to find the info they want, and 20% of those people are giving up. Not exactly the kind of thing you'd want to hype to investors. Of course this is an example, and there could be a variety of reasons that create these numbers, some good, some bad.  The important takeaway is that we don't know or understand what is happening.

In this case your excitement about your endeavor led to pride & admiration regarding your early traction (and rightfully so!) which ultimately led you to tracking irrelevant data. On to definition number 2:

Worthless and Futile

The second most common problem with vanity metrics is that they are basically worthless, and possibly futile (don't worry, there is light at the end of the tunnel). Vanity metrics can not be acted upon or provide insightful information about the true nature of a company's interaction with its users/customers. Let's take pageviews for example. What exactly is a pageview? What does it tell you about the behavior of the people that are using your product? Nothing, just that they looked at some pages on your site. In the words of someone smarter than me, who wrote way back in 2009 and expanded on in 2014's:

"Metrics are people, too" - Eric Ries

What does he mean by that? Pretty simple really. Metrics are merely the representation of the behavior of real people using your product. Revenue? People giving you money. Pageviews? People clicking around on your site. Churn? People giving you the proverbial finger. The commonality here is people, not numbers. So tracking something like how many times someone clicked on your site gives you absolutely nothing about why they are clicking your site, how they got there, or if they are going to become a customer.  We need better information to understand these issues but don't worry, there are plenty of ways to do so!

The promised light at the end of the tunnel

So we have covered a bunch of doom and gloom scenarios and started to consider what to track, and more importantly what not to track. Let's take a look at some common vanity metrics and how changing your perspective can uncover the real value, an actionable KPI.

 (DISCLAIMER: These vary from situation to situation, and company to company. It is incredibly important to understand your company as you look for what is a vanity metric and what is a KPI). 

Vanity Metrics

• Trial Users

• Page Views

• Social Media 'Likes'

• Email Subscribers

• Leads in Sales Funnel

• Marketing Spend

• Total Customers Acquired

• Monthly Revenue per Customer

vs. their Actionable KPI counterparts:

Actionable KPIs

• Converting Users

• Conversion Rate

• Social Media Engagement/Referrals

• Email Opt-In Conversion Rate

• Cohort Analysis of Sales Funnel

• Return on Marketing Investment

• Customer Acquisition Cost

• Customer Lifetime Value

Let's take a look at the thought process behind this shift in thinking. Let's take Trial Users > Converting Users. For most companies, total trial users is an exciting metric! This is a list of people who want to use your product and are willing to try it. However, we know now what to look for in a metric. The number of trial users does not tell you anything about the behavior of your users, nor does it tell you if the product you have is what the users want to pay for. An example would be having 2,000 trial users. That's fantastic! But if not a single one stays one past their trial, you turn them into zero customers, which is pretty shi... let's just say less than fantastic. Looks like we identified a vanity metric.

So in order to avoid this, let's look at what we should track in this example. The real KPI that matters is how many people stay after their trial and ultimately pay for the product. So, we could reason that the true KPI would be the number of users converting to paid plans. So let's say we have that 2,000 trial users and 1,000 convert to paid customers after the trial. Now we are getting somewhere! We know we have a 50% conversion rate, and we know we have two groups of users (those who converted to customers and those who did not). We can take action on this, reach out to both groups and ask why they converted or opted not to convert. We can then adjust our product, our marketing, or our customer segment in order to increase the actionable KPI, which is the conversion rate of trial users. Note we are still tracking trial users, this is an important component of understanding our product, but it is not the metric that will define our success.

I wish it was as simple as setting up a list of "track this", "don't track these". However every business is different, and the metrics that represent user/customer behavior are different for all. On top of that, these metrics evolve! After all, no sense fretting about revenue when you are 6 months from having a product in the market.

What is important is to consider how people navigate your product and the quality of the data you receive about their experience. And there are lots of people working around the clock to help you do this! In fact, I know of one such company that is making this a cornerstone of their business. How's that for a rockin' (shameless) plug.