Tag: metrics

Download Requests Alone Are Pretty Useless

Post ImageBack in December I said that podcasters should gather and share as much statistical data as possible. It is now exactly three months later and, surprise-surprise, I still feel that way. The topic resurfaced on Saturday when Adam Curry from PodShow posted about the number of download requests the PodShow network has received:

For the record, In December 2006 the network produced 52 million download requests.

Earlier today, Paul Colligan picked up on the story, and said:

Yes, Virginia, Podcast download numbers are important. And these are important numbers.

Yes, they are important. And 52 million is an impressive number, especially for just one month. But stating download requests alone is useless. I think they are important to track, but you need to pair them with something else. Something like a completed download.

Understanding the difference between a download request, a partial download, and a completed download can be somewhat confusing, so let’s try this metaphor on for size.

Download Request = Walking into the grocery store with a shopping list in hand
Partial Download = Filling your shopping cart with some of the items on your list
Completed Download = Filling your shopping cart with everything on your list
Partial Play = Eating some of the things you purchased
Completed Play = Eating everything you purchased

It’s important to note that each one is harder and harder to measure. It’s fairly trivial to count the number of people entering the store. It’s a little harder, but still fairly easy to count the number of people who have at least something in the cart. It’s much harder to count the number of people who have everything on their list. And to count the number of partial and completed plays? Much, much harder.

When you have all of these statistics, you get a really nice picture, don’t you? You can see a nice comparison of how many people entered the store and how many people actually bought something. Having just the number of people who entered the store is not that useful though. Sure it has some useful implications – if tons of people come in but don’t buy, maybe we don’t have what they want, or they didn’t like what they saw, etc. That makes sense in the offline world. In the online world though, it’s not just people that make download requests – it’s other applications and services too. It’s like counting every person, stroller, cart, bag, and box that enters the store – and clearly, only people can purchase something! To make matters worse, some applications will make multiple requests for a single download. Imagine a person walking into the store, putting something in the cart, exiting the store and walking back in again to put the second item in the cart.

So download requests are interesting, but they are really only valuable alongside other information like completed downloads. Or a list of which applications made the requests (this would allow you to tell the difference between a person and a stroller, for instance). Citing just the number of download requests will do nothing but mislead.

Podcasting Metrics: Complete Downloads & More

Post ImagePodcasting consultant Jason Van Orden has been writing an interesting series of blog posts on podcasting metrics. In part 4 of the series, Jason tackles the issue of measuring complete downloads, and says that he doesn’t think measuring complete downloads is “absolutely necessary” and that something “more sophisticated and qualitative” is needed in addition to download numbers.

From part 4b of the series:

Scott Bourne and Tim Bourquin provided interesting and relevant responses. They both emphasize that podcasters have a responsibility not to let advertisers hold podcasting to a higher standard than other media (i.e. magazines and newspaper) that can’t measure complete content/ad consumption.

I have to respectfully disagree.

The way that magazines, television, radio, and other media sell advertising is flawed. Everything is based on assumptions (circulation numbers in the case of magazines, random sampling in the case of TV and radio). Don’t think for a second that advertisers are happy about this system – I’m sure they’d love to know exactly how many people watched or heard or read their advertisement. Why do you think everyone loves AdSense? Cold, hard numbers. The problem with magazines, TV, and radio is that the technology to accomplish this is prohibitively complex and expensive.

Podcasting doesn’t suffer from this problem. Measuring exactly how many people have downloaded an episode is relatively straightforward and inexpensive, and while not 100% accurate, it is fairly close. I think the strategy that Scott and Tim suggest would be bad for podcasting. As the saying goes – you’re only as strong as your weakest link. Podcasting needs to be stronger than other media.

A Better Strategy

I think podcasters who wish to generate advertising revenue should provide as much data as possible, even beyond complete downloads if such data is available (more on this in a second). There are a number of reasons for doing so:

  • There would be less waste, as advertisers could spend money only on podcasts that generate views or listens of their ad.
  • More data could also allow advertisers to more appropriately target their ad, making it more effective, enjoyable, and useful.
  • In the long run, advertisers would move more dollars away from media that uses flawed assumptions to media that provides useful data. That is, podcasting’s piece of the advertising pie will grow.
  • The valuation of a particular podcast will be much more realistic.

I am sure some podcasters are bristling at my suggestion. They think that if they have to provide actual numbers, they can’t make as much as if they sold ads based on assumptions like the other media do. This idea is wrong too. Providing more data will allow advertisers to spend targeted dollars. Unlike general advertisements, an advertiser will pay much more for the ability to target an ad. The podcaster may actually end up making more money!

Podcasting’s enemy (if we need to have one) is not the advertisers as Scott and Tim suggest, it’s the other media. Give the advertisers what they want, and podcasting will prevail.

Beyond Complete Downloads

I think complete downloads are quite important. We are putting the finishing touches on a big update to Podcast Spot, and one of the new features we have added is complete downloads. We parse the request logs for you automatically, so you’ll see the number of complete downloads for each episode, usually within two hours of the download. Right now these numbers are best effort, meaning that we aren’t yet at 100% accuracy. We’ll continue to work on it though.

As I mentioned above, podcasters should strive to provide as much data as possible to advertisers. There are the obvious things like complete downloads, page views, geolocation stats, demographics, etc. There are also the less obvious things. What if you could determine if someone actually listened to or watched your entire episode, or if they skipped parts of the episode? That kind of information would be extremely valuable.

These are the types of metrics that podcasters should strive to measure. Podcasters don’t have a responsibility to hold podcasting to the low standards of other media, they have a responsibility to set the bar higher and higher.