Dr. Clay Christensen, author of The Innovator’s Dilemma and inventor of Disruption Theory, is one of the rare people to be revered as a Business Prophet by both tech leaders and management consultants. The Innovator’s Dilemma is required reading in both camps. And an awful lot of smart people, who’ve really walked the walk, swear by his writings as foundational text for how the tech industry works.

But there’s a problem. Christensen’s public predictions about tech companies, to be honest, have been real stinkers. He famously predicted that the iPhone would fail; everyone’s heard that one. But there’s also Uber, and MOOCs, and if we go back far enough, most of the companies in his own book. Perhaps most frustratingly, his most actionable advice – Job to be Done theory – has found better adoption in hired gun management consulting than it has in actual product development, or in actually disruptive businesses.

There’s something that needs be reconciled here. Disruption Theory describes something real, but it’s generating bad advice.

It feels to me like there’s something interesting behind this, and no one has really articulated why. I took a stab several years ago about how Apple repeatedly escapes disruption, and Ben Thompson wrote a good piece on that subject a few years later. I believe what we wrote happens to be correct (“continually push expectations upwards, and never let customers become overserved”), but it’s a bit of an obvious platitude. I don’t think either of us cracked open the real mystery.

Ben’s most well-known piece in this area, What Clayton Christensen Got Wrong, never inspired me all that much. “Consumers aren’t rational and they like nice things” doesn’t scratch the itch; to me it sort of cheapens the insight of the theory (which, I believe, ought to logically hold true everywhere, not just only for enterprise as opposed to consumer) and for a while I really struggled with how to untangle this.

After thinking about it for a long time, I have an idea of what’s happened.

In today’s tech industry, Disruption Theory gets the process right, but it gets the resolution level wrong. It brilliantly describes a set of forces and incentives that really exist, which is why it resonates so much with people who’ve walked the walk.

But today, those forces are playing out on a level that’s different from how Christensen initially understood them. Today’s disruption isn’t a story of individual businesses disrupting incumbents, but rather of business ecosystems disrupting incumbents. The result is a framework that feels more relevant than ever, but that generates reliably incorrect advice.

Let’s figure out how this happened. First of all, to really understand the Christensen, Inc mindset, we need to look at how disruption theory fits into what is really a three-part worldview:

The first Christensen Idea is Modularity Theory. Modularity theory starts with a familiar categorization of how all of the different components of a product or service interact with each other: predictable and independent (modular) versus proprietary and interdependent (integrated). Modularity theory predicts that as competitive environments mature, the initial performance and predictability advantage of integrated systems eventually lose ground to the cost and compatibility advantage of modular systems.

The second one, which is the famous one and came first chronologically, is Disruption Theory. Disruption Theory explains a process through which incumbents who do “all the right things” get unseated by upstarts anyway. Incumbent businesses (who often sell integrated products that command high profit margins, and prioritize “sustaining innovations” that increase performance) periodically come under competitive pressure from new entrants at the lower end of the market. These new competitors often use cheaper, modular, good-enough technology that can pick off undemanding customers who are overserved by the high performance and high cost of incumbents.

The Innovator’s Dilemma goes through three case studies of disruption: steel mills, construction equipment, and disk drives. In each case, a certain critical mass of customers inevitably become “overserved” by the high-end performance that incumbent solutions offer. The first important part of the theory is understanding these customers, for whom the delta in performance between a high-end integrated solution and a lower-end modular solution is unnecessary. They become good prospective customers for new entrants.

The second important part of the theory is its explanation for how incumbents respond to this kind of threat. Christensen explains how the rational business decision for incumbents is to retreat upmarket rather than compete head-on against these disruptors, because entering into a low-end competition undermines their own business model. This could be for a variety of reasons. Management could feel that their integrated product, and the high quality, brand, and high margins it bestows, is too important to compromise. It could also be that the business’s cost structure or debt structure does not allow them to compete on price. Either way, the hallmark of disruption is seeing incumbents have “allergic reactions” to this new form of competition.

The third idea, which is the one that offers the most actual advice, is Jobs-to-be-Done Theory. JTBD theory is really a mindset exercise for how to step into your customer’s shoes and understand what elements of performance they actually care about in a product, versus which elements they don’t.

The classic story that gets told here is the Milkshake Story. Christensen’s group was hired by a fast food chain who wanted to sell more milkshakes, and wanted to better understand their competitive standing compared to other desserts like ice cream. When they were surprised to learn how many milkshakes were sold between 7 and 9 AM, the group realized that customers were hiring milkshakes for a different job than they’d previously believed: the job wasn’t “reward me at the end of a hard day”, like a dessert, but rather “keep me fed and happy during my boring morning commute.”

The moral of the milkshake story, which is broad but correct, is basically “know your customers”: they may care about your product and its features for a different reason than you assumed. In this case, the most underrated product features of morning commute milkshakes had been their portability and thick texture, since they were being consumed in cars over long drives. In the Christensen worldview, the key lies in understanding the JTBD of overserved and restless customers: people currently buying a product they can’t take full advantage of, and who are both less demanding but also more willing to switch to a new entrant.

At the intersection of the three theories, you find the core of the Christensen worldview, which is a theory of customer choice. It basically goes:
“Consumers know what jobs they want done, and they’re going to hire products that get that job done as cheaply and effectively as possible. High-end incumbents with integrated products often fail to recognize or are unable to respond to the threat posed by lower-end, modular competitors who can solve users’ needs.

The solution is to step into your customers’ shoes, and understand their JTBD as your guiding lighthouse for navigating your product’s purpose amongst the competition. If you are a disruptive business, then you compete on cost and modularity, and you win by replicating the JTBD of overserved customers. If you are a sustaining business that competes on performance, then the incumbent wins.”

How applicable is all of this? Clearly, disruption – as more or less originally described by Christensen – is happening in the big picture. Technological eras and big disruption narratives are defined, in big terms, by overserved and opportunistic customers shifting to new platforms and offerings while incumbents are paralyzed from responding by their own business models:

• Expensive, integrated IBM mainframe computing systems were disrupted by modular PCs running inexpensive microprocessors (Although IBM famously built their own PC in a completely separate location, a notable validation of The Innovator’s Dilemma)
• Expensive, integrated IBM mainframe servers were disrupted by Unix on Sun boxes, and then Linux on x86 racks
• Expensive on-premise IT departments are getting disrupted by cloud computing and web-based software-as-a-service
• Expensive PC fleets in controlled environments are getting disrupted by bring-your-own mobile devices and App stores
• Expensive publishers, distributors and gatekeepers are getting disrupted by aggregators and plug-and-play direct-to-consumer platforms
• Expensive, integrated car ownership is (one day, probably) getting disrupted by modular transportation-as-a-service

These are big shifts, and they are genuinely disruptive: incumbents are paralyzed from responding by their own business model, which requires them to sell customers something they no longer (or perhaps never) needed. The incumbents look like classic disruptees. The problem is that fewer and fewer of the newcomers look like classic disruptors. Upstarts are competing on performance and going after high-end customers, which isn’t at all what disruptors are supposed to be doing.

Instead, we’re seeing ecosystems of many different products and businesses, many of which in isolation look like integrated businesses or a sustaining innovations and compete on high-end, quality user experience. But as they do so, they create an ecosystem of technology that undermines and eventually disrupts the integrated, single-purpose legacy systems that preceded them.

Let’s look at one example of an industry that’s been disrupted by the smartphone ecosystem: the consumer electronics industry. Digital cameras, video recorders, and other kinds of multimedia equipment were fairly integrated, relatively costly, and competed largely on performance. (Remember the megapixel wars?)

Now consider how they’ve been upended by the combination of mobile phones with cameras on them, and apps like Instagram that do something with the camera. Neither the iPhone nor Instagram are pursuing disruptive product strategies. They obsess over user experience, and started with the most demanding customers in their markets.

Nor do they do the same kinds of jobs to be done as the handheld electronics they replace. The smartphone’s job is “An interface on the whole world, in your pocket” and Instagram’s job is “Find out what your friends are up to, share and signal status”; neither of those compete directly in a JTBD sense with the integrated electronics that came before them. If they had, Apple would look like Sony, Instagram would look like Flickr, and neither would be worth hundreds of billions of dollars.

But the combination of the two was hugely disruptive. They compete with a technological and cost advantage that gives traditional electronics vendors no way to respond other than by retreating to the high end, like DSLR and 4K cameras (which still sell). The combination of the two is also quite modular: iPhones can be swapped out for Android phones, Instagram can be swapped out for Youtube, Vine, Snapchat, or TikTok, and they all work together predictably. The outcome is disruption, even though no individual participant’s playbook is.

Here’s another example: the taxi, limo and rental car industries, who for many years competed with one another in a relatively stable arrangement. But then we started to see some apps, most notably UberCab, who started as what I’m sure anyone would call a sustaining innovation: a way to hail black cars faster. On its own, that’s not disruptive.

But when you pair it with mobile connectivity via smartphones and democratized navigation superpowers via Google Maps (also, not really a disruptive product in isolation), you create space for this new, modular ride-hailing ecosystem (swap out Uber for Lyft, iPhones for Androids, one driver and car for another) that created something actually disruptive: UberX, a labor platform against which incumbents can only really compete via regulatory appeal or by retreating upmarket. The taxi business won’t compete in the broader labor market for willing drivers, and rental companies don’t want to compete against cars people already own; the incumbents have all the right allergic reactions you expect. Disruptive outcome, but made out of high-performance, “sustaining” parts.

Compared to the legacy systems that preceded them, this is modular competition! The tech community is continually reshuffling itself, finding new combinations of apps, endpoints and real-world assets that serve users in new, creative ways to remove friction and bypass incumbent cost structures. The whole mobile ecosystem, starting with the most demanding customers, is clearing space for new business models and product offerings in a way that incumbents have a hard time responding to. It’s disruption, except the disruptors don’t look like disruptors.

That’s key to the mystery: the ecosystem as a whole may be disruptive, but any given piece of this ecosystem does not really want to compete on price or plug-and-play modularity at the low end. They want to deliver the best experience possible, not compete on cost! They want the best users, not the lousy ones!

This may be part of the riddle as to why sometimes disruption seems to “spread from above”: because the conditions for disruption appear when some critical mass of new tech companies get adopted by early users, and most of the time, those are going to be the rich power users who make for great customers. But once you have a critical mass of mobile ecosystem, there’s enough capacity to do actual in there for customers to start abandoning legacy solutions.

Christensen devotees might at this point protest, and argue that these companies represent “New Market Disruption”. In retrospect that’s what they were, but in practice that’s not what they looked like. They looked like sustaining innovations! The iPhone was famously lambasted by Christensen as a sustaining innovation on the cell phone. Uber was criticized as a sustaining innovation on livery cars. Almost everything starts out as a derivative of something else that already exists, addressing a market that already exists in some form. If Disruption Theory can only recognize New Market Disruption in retrospect, then it’s not that useful.

The thing is, just to repeat this again, Disruption Theory is doing a great job of explaining the forces and incentives that exist at the level of the ecosystem. So it rings true, as it should. The issue is that it’s explaining a kind of gameplay that’s carried out at one level of abstraction higher than what the product and growth people at disruptive tech companies actually care about, or what early-stage investors can measure and finance. So the advice that it spits out feels as though it should be true, but ends up being paradoxically backwards. Now we can understand why.

By the way, the Innovator’s Dilemma isn’t the only example of a well-loved tech theory that gets a process right, but the abstraction level wrong. The Lean Startup is another one. The Lean Startup described a worldview about innovation that preached, “We don’t know what works, we have to try everything, so run lots of little experiments and iterate on your position rapidly in order to figure out how we ought to build the future.” That feels right, and it is right, except it’s right on the level of the startup community as a whole; not on the level of any given company.

The innovation economy works because it’s full of startups, each of which represents a low-cost experiment seeking validation. However, if you are one of those experiments, your best shot at succeeding isn’t to surf around and do a lot of low-commitment experiments and try to iterate towards an idea that works – it’s to obsessively pursue a vision of the future, use every possible tool available to you to build it, and not quit until you’ve changed the world or your business has died trying. The ecosystem collectively benefits by being lean, but any given participant shouldn’t strive to be.

It’s a similar story going on with Disruption Theory, I think. Disruption is happening, and the tech industry is having a dynamic and revolutionary impact on industries around the world for precisely the reasons that Christensen articulates in his work. But the units of resolution aren’t right; maybe not everywhere, but that certainly feels to be the case in today’s tech world.

Like this post? Get it in your inbox every week with Two Truths and a Take, my weekly newsletter enjoyed by thousands.