The first season of Serial is nearly over. I’m sure we’ll get to read more pieces discussing how many people have been listening to it, and some general hypothesizing about what those numbers mean for the future of podcasting.

This post explains why we have no idea how many people are actually listening to Serial (or any podcasts).

Let’s start with two games of Spot the Difference:

1. “Serial has been downloaded or streamed on iTunes more than five million times.” The Guardian, Nov 18
2. “Serial… has garnered the rapt attention of over 5 million downloaders.” The Guardian, Dec 11

and

1. “Serial averages over 1.5 million listeners an episode.” New York Times, Nov 24
2. “The team behind the podcast told CNBC that they were averaging 1.5 million downloads an episode.” CNBC, Nov 21

Figured it out?

Listeners ≠ Listens ≠ Downloads

Let’s dig deeper…

Five million downloaders… or downloads?

Downloads are very different from downloaders. One downloader can download multiple episodes, multiple times. For example, I’ve downloaded all eleven episodes of Serial, a ratio of 1 downloader to 11 downloads.

In late November, Apple reported that Serial was the fastest ever podcast to reach five million downloads or streams (I think this is the number that people are referring to when you see “Serial” and “5 million” in the same sentence).

The number should be taken for what it is, a hard number of downloads and streams in iTunes. It doesn’t actually help us to learn much more than that:

• There are many other ways to download and stream Serial (as with all podcasts) including (1) other mobile apps (both iOS and other platforms) (2) other desktop apps and (3) the show’s own website. Of course, far more than 5 million downloads and streams of Serial have taken place. The Serial team probably knows roughly how many, if they’re logging and counting file downloads from their servers.
• We can’t really compare it to anything. Apple provides podcast producers with very minimal data, and certainly nothing useful to benchmark against. The Guardian cites Apple as saying that iTunes and iOS users will “listen to 7bn podcasts in 2014 alone,” which actually makes the Serial number of 5 million seem puny (.07% of all podcasts, maybe it could hit .1% by the end of the year).

Listeners or downloads?

David Carr doesn’t provide his source for the “1.5 million listeners” in his column. But the same number crops up in a CNBC article, though this refers to “downloads” not “listeners.” The source is given as the “team behind the podcast”.

It’s easy to confuse downloads and listeners.

They differ in two very important ways:

The same listener can download multiple copies of the same podcast. For example, I personally have downloaded each episode of Serial at least twice, in some cases three times:

• I switched podcasting apps a few weeks ago, and the new one downloaded the entire series, even episodes to which I’d already listened.
• I have an old iPad that still has a podcasting app installed, happily downloading episodes to which I never listen.

In web analytics, we call this duplication. It’s a big problem for counting users (often referred to as “uniques”) on web and mobile: the same person can read something in a mobile web view and later on their computer; it’s very hard to relate those two sessions as belonging to the same person.

Just because something is downloaded, it doesn’t mean that anybody ever listened to it. I think this is incredibly important. I only just switched podcasting apps, and I already have 20+ episodes downloaded to my phone, to which I’ll probably never listen. My old app must have had over 200 episodes that hadn’t been listened to. I would guess that I listen to roughly 10% of all podcast episodes that I download.

If everybody was like me (I don’t think they are), then real listening numbers would be as little as 5% of actual download numbers.

So, why don’t podcast companies report listens and listeners instead of downloads? The simple answer is that that data is locked up inside the clients that play the podcasts to you. Perhaps some of them are working on making it available, but (as far as I’m aware) none of that data is making it back to podcast producers and publishers. At least, not yet. Nobody really knows exactly how many people are listening to podcasts.

Of all the companies that could conceivably have access to some estimate of actual listening (vs downloading), it’s Apple. They manufacture listening devices, and built the operating system. But when they claim that 7bn podcasts will be listened to this year, I’m skeptical. If they really had actual listening data, it would be surprising that this was the first we’d heard of it. I assume that this really means iTunes and iOS users will download or stream 7bn podcasts. That would seem at least an order of magnitude more reasonable. It also makes the Serial listenership (for which, I’d argue, there’s a higher percentage of downloaded episodes listened to) more impressive in the greater picture.

Even if we knew how many people pressed play, we still might not be satisfied. It’s a similar problem to that of readership on the web (something I’ve been working on for a while): just because someone viewed the page with a story on it, it doesn’t necessarily mean that they read much of it. So when does a podcast listener actually qualify as “listening” to a podcast?

I really am excited by the recent successes of “Serial” and “Startup”. But I’m nervous about the hype. Radio is a medium that’s dear to me, and I’d so love any resurgence in digital audio to be real. Like Jason Fried, I believe that the real magic in both of these podcasts might be the return to serialized storytelling… they’re compelling and addictive. That wasn’t technology’s doing, just that a couple of radio geeks decided to use the medium they know so well to tell their stories.

There must, at some point, be a bigger migration to digital on-demand audio. Better programming will be a big part of that, but decent data (and its responsible usage) is vital as well.

Cover photo: Flickr user Jake Bellucci