Picking Metrics That Matter!

When it comes to analyzing data, it's really important to pick the right numbers to focus on. And a lot of companies get this wrong, a lot of products, a lot of websites. It's really hard for some people to avoid these mistakes. And so I wanna run through a couple of things here that really should help you understand when you're analyzing data, how to pick up the right metrics in order to achieve your goal.

First, we're gonna talk about process mapping. Then we'll get into the different types of metrics that you should consider.

Process mapping is essentially just laying it all out, putting all the parts of a given business process on a map and seeing how they interact. This will help you then figure out what metrics you need and how to improve that process. Let's take YouTube as an example. YouTube has a business model in which you have YouTube creators like myself and millions of others, making videos for people to view. So on one side you have the creators. On the other side, you have the viewer. In the middle, you have the platform. Now, this would be a very harmonious, beautiful thing, except there's no money involved here and so this wouldn't be sustainable. So of course, what YouTube does, it works with advertisers who put the ad in front of the video, which is how they all make money. So from there, the viewer can choose to skip the ad or let's say it's something they actually want. They will click on it, engage with that ad and money will be exchanged. A large portion will go to YouTube and a smaller, much smaller portion will go to the YouTube creator. Fun fact, it takes about a million video views per month to make around $2,000 to $3,000, just the basic ad rates on YouTube platform right now. So this is the most rudimentary fundamental part of what I would call a process map. You have all the different parties here. You have the people creating the product, the platform facilitating the transaction, the advertisers providing their ad platforms and the things that they wanna sell to the viewers who then engage with them and then everybody kind of gets paid out of that.

All in all, this is a very simple process, but one that you could say describes YouTube's entire business model. Of course, there are a million processes in each one of these major steps. However, from a very high level, this is essentially how YouTube works.

The first part of this process is publishing, where the YouTuber will create the video. This technically happens before it gets to YouTube, but it is no less a part of this process. And YouTube does have ways to help YouTubers perform this step. Then they upload that video to YouTube. And finally, they publish the video on YouTube. A lot of videos will remain in sort of an unlisted or private state until the encoding and processing, tags, title, description, all that stuff is really just ready for the YouTuber to publish. So there is a gap in time, it's not just immediately upload and publish. On the other side of this process, you have the viewing where the user views the video. In addition to that, the user also has to log in, the user can resume a session. Let's say you pause a video, come back, resume play maybe on a different device. There are lots of things involved in just actually showing the user the video. We can think of it in these kinds of simplistic terms, but there are trust me, are a mountain of things that take place from the time you hit play on the video to the time you start seeing an actual video. So the third part of this process, the way I would chalk this up is as engaging where the user can click on an ad. That's obviously the thing that keeps the whole system running. They also can share the video. It's a great way to increase awareness or traffic to that video. Users can also comment and they can do likes and dislikes as well as maybe a few other things, clicking on cards, end screens, subscribing, et cetera, et cetera. So you can think of this whole process, that very simple map, as these three categories of things. And the way you often can describe a business process or one of these steps, as an object taking an action. You notice every one of these statements has an object, a person, in this case, a YouTuber, a user doing something: creating a video, uploading a video, publishing a video, playing a video, clicking on something, sharing something. So this object action framework is a real common one that you can use to describe business processes. And when you do that, the outcome is a list of metrics and things you can track.

For example, how many uploads were there? How many views were there? How many clicks on an ad were there? All of these little tiny object action snippets or steps can result in a list of metrics for you to use in identifying how to make a business process better. But of course, not all metrics are created equal. In fact, there is a big disparity in these two major types of metrics that you're likely to either see or analyze yourself. The first one are vanity metrics, and we'll get into what that means in a second here. But these are fairly useless metrics, and you'll see why. Then you have the complete opposite side of that, actionable metrics, numbers that tell you something that you can act upon. This is what we should always be striving for because simply looking at the vanity metrics doesn't help us make decisions or improve anything. And so this is essentially the spectrum. On one side vanity, on the other side actionable. We can't always get exactly the actionable metrics we want, but we should try to get as close to metrics that help us in this way as closely as possible. So let's take a look back at our process here. You have the publishing side where the YouTuber is performing these three basic actions. Then you have the vanity metrics associated with each one of these. Let's say the first one would be the total videos on the platform. These were how many videos the YouTuber created. Then the number of uploads on the platform ever in all of time. It's a very large number. And lastly, the total public video or the ones that have actually been published and are viewable. So these three metrics can help understand these processes.

They do describe these steps that the publisher, the content creator is taking. However, they don't really tell us much. So if we wanted to get into something deeper and understand maybe a way to use these actions and measure them that helps us make decisions or understand how things are going, we would create these set of metrics. So first off, instead of just the total videos on the platform, we should look at the growth of the total video. So when we think about the number of videos, we shouldn't just look at the overall number. We should see how that number is actually growing or shrinking in terms to a previous period, let's say last year the same month, et cetera. You can also do this for uploads, where instead of looking at overall uploads, you look at uploads for a given timeframe, let's say, day. And then you look at the average uploads per day, let's say over a seven-week moving average, and you see how today or the current day does compare to that seven-week moving average. Is it up or down? Are people uploading more videos today than they have on average for the past week? Actually pretty interesting, right? That's a number you can look at and go, hmm, why is that? You can trend it over time and it can tell you things about times of year, seasonality, et cetera, et cetera. And lastly, let's look at the time between upload and publish.

As I mentioned, a YouTuber has a lot of steps that they need to do before they actually publish the video. You can just hit go and make a video public from the moment you upload it. But typically what you'll do is you'll upload the video, you'll add a really good thumbnail that helps describe the video so someone sees it, they want to click on it. You'll give it a good title, give it a good description, add tags, add info cards, add an end screen. There are about 35 steps, I measured last time I did a video, that adds to the time between when you upload and then you publish a video. So we could measure that and we could trend it over time. And that would tell us from a YouTube standpoint how good or how much work is involved in between the time you upload to the time you publish. And let's say we wanna reduce that, this would be a way to measure if our efforts are fruitful. Let's look at a basic dashboard here, and this is the most rudimentary dashboard I could possibly imagine for YouTube, but it illustrates this point effectively. Let's look at the total videos. This number is probably incredibly small, it's probably in the trillions, but let's assume that total videos ever on YouTube are 300 billion. Well, guess what that number will be tomorrow when we come in? It'll probably be 300 billion still. Now if we added a decimal, maybe in a few weeks it would go from 300 to 3.1, 3.2, 'cause the number is so large, this is an incredibly meaningless number. However, you'll even see companies like Facebook or Google or these guys quote numbers like this frequently for some sort of splashy effect in the media. But from a business standpoint, this is utterly meaningless. Then you have total users. Now, when I was working at Facebook, we actually did a query on the user's table. And it was something like 13 billion people. Well, of course, there are not 13 billion people that are alive in the world, so this number couldn't be true. So if you look at the total number of users ever, it again doesn't tell you anything. And over time, this number is gonna be relatively stagnant, it's gonna remain the same here. And then of course, let's look at hours watched: 800 billion. Again, probably very, very low in reality, but the same principle applies. These all are pure vanity metrics. And you may hear executives ask for these things from time to time. "I wanna know how many users we've ever had, how many people have ever visited our website." And then the question would be why? What are you going to do with that number? How is that gonna help you do your job better? Whatever that is, marketing, sales, et cetera. And that's where I think if we were just to look at these three numbers, we could come up with a much more actionable, useful dashboard. And here's my first attempt at that. So my first attempt here at a more actionable version of this dashboard for YouTube would start with monthly uploads. And this is similar to the total videos, however, it's time-bound to a given month or we could do a rolling 30 days, however we defined a month for this. So it still is this large number that over time people will begin to understand if that's a big or a small number. But notice right below it, we also list a growth rate, the actual change in this number from a previous period. Often, we would do from the previous month. So here you see something that helps people talk about things in the grand scale, but also you get something actionable. You can tell is this going up or going down, which is key. Next, I could add monthly active users. So instead of just looking at overall users, essentially anybody that's ever watched a video on YouTube, which is some absurdly large number, I can see for the past month how many people logged in and viewed a video. Now, defining what an active user is and what even a month is can be very long processes for a lot of businesses. But this at least is something that is much more actionable and focused on the current state of things. And again, we have a large number there, 1.63 billion, and then we have a growth rate, again, seeing is that going up or going down? So if I'm YouTube and my goal in life is to get people to watch videos forever and ever and get more people to do so, this is a very actionable thing, just a quick check to see are we growing or are we shrinking? Next, we can see average session time. And this in terms of hours watched, the previous example, this one is gonna tell me how long are people spending on the platform on average. Again, we have 1.63 billion users, a ton of users looking at a increasingly large catalog of videos. However, the key thing is are they spending more or less time on the platform? We could have an endless supply of videos as YouTube essentially does, but if people aren't engaging in the right way and staying on the platform, then we're not fulfilling our job of giving advertisers people to sell to and giving creators a way to monetize their business and their content production.

So when it comes down to it, the goal is to figure out what business process you're looking at. What are the steps in that process, object and action? How can you measure those things? And then what are the rates in actionable metrics that you can tie to each one to help the people within those processes do their job better, or for the very minimum amount, they can understand if their efforts are helping the growth or decline of their process. It doesn't have to be a metric tied to a given action, meaning we did a marketing campaign and we saw sales go up. It can just be an overall metric for that process, but one that we can use as kind of the benchmark of, okay, if all of our efforts, the sum of all of our marketing efforts, did they increase the click-through rate on our email campaigns or our daily signups? Again, things that tell us a kind of a barometer, a temperature check of like how it's going day by day, not this overall big vanity metric, like total number of users ever, total sales, those kinds of things.