• × mine mccobbs post for more? Strategic planning models a better approach to the Traditional Approach…

Predicting markets

Analysts at Goldman Sachs spend their whole life learning advanced mathematics, building sophisticated models of complex corners of the world, and are financially incentivized to predict the future accurately, because Goldman Sachs makes billions of dollars if they can be 10% more correct than the next firm (who also employs brilliant analysts).

So, how accurately did they predict economic metrics within their area of expertise? They got it very wrong, for 25 years, often not even directionally correct:

Here’s another:

But maybe the macro economy is too hard to predict. It is “chaotic,” we are told, in the mathematical sense that small changes in inputs result in large, unpredictable changes in outputs. The “Butterfly Effect.”

When we create strategies, we’re told to “skate where the puck is going.” That means predicting the future, such as macro economic trends, and the trends of our industry and immediate competitive markets.

But if the smartest, most motivated experts can’t do that, why do we think we can?

Let’s try something closer to what corporate strategists must do: Predict the future for products in well-understood industries.

Predicting sales

Researchers at McKinsey & Company, a consulting firm, studied the sales forecasts for new drugs made by analysts at brokerage firms. The data included more than 1,700 individual forecasts on 260 drugs over a recent ten-year span. … The average error was large. Almost two-thirds of the estimates missed the peak revenue amount by 40 percent or more. Further, forecasts for follow-on drugs were no better than the first launches within a therapeutic class.
—Michael Mauboussin & Dan Callahan in Total Addressable Market, Credit Suisse 2015 (my emphasis)

Once again, highly incentivized, highly trained analysts, within their area of expertise, within a well-understood, highly-regulated industry, mostly get it really wrong, whether predicting sales of a new drug, or predicting sales of a new competitor to an existing drug.

Mauboussin and Callahan suggest that one way to increase the accuracy of predictions is to “use base rates as a reality check.” That is, use the industry average as the starting point for your prediction.

Does this method work? They illustrate:

Here’s an example of how the base rate approach can figure into your judgment of TAM. During a conference call in February 2015, Elon Musk, the chairman and chief executive officer (CEO) of Tesla Motors, suggested the company might be able to achieve a 50 percent compound annual growth rate of sales for the next decade …

… The base rate method … simply asks: “What happened to other companies when they were in a comparable position?” Exhibit 12 shows the distribution of 10-year sales growth rates for more than 1,200 instances of companies of a similar size as Tesla is now, measured by sales. The average growth rate, adjusted for inflation, is less than three percent, with a standard deviation below eight percent. Further, no companies achieved a compound annual growth rate (CAGR) in excess of 40 percent … We have placed a star at the growth rate Musk mentions.
—Michael Mauboussin & Dan Callahan in Total Addressable Market, Credit Suisse 2015 (my emphasis)

As we see from Exhibit 12, Musk’s call is completely outrageous when using the base-rate method:

It is 2023 as of this writing, so we can evaluate Musk’s call and the base-rate method:

So, the base-rate method is suspicious, and the analysts are suspicious. Is anything not suspicious?

When we build strategies, we’re predicting what sorts of products are going to sell, often inside a fast-changing market, selling to customers with fickle desires in an ever-changing competitive landscape. Drugs that address a well-known disease with well-known rates of incidence and trends, with well-known competition, are easy in comparison. Easy, but already almost impossible even by the experts. And using base-rates doesn’t give us much confidence either.

So why do we think we can do it?

Predicting winners

Chess Grandmasters spend far more than 10,000 hours reaching an elite, unbelievable level of skill. They instantly pattern-match board positions against thousands of games they’ve memorized. They have an intuitive sense of how the future might unfold, without actually thinking through every possibility. They understand the relative strength of the best players, because they’ve studied their games for years. So when they guess who is going to win the 85th annual Tata Steel tournament—arguably the most prestigious tournament on the annual circuit—it is more than just “sports betting.” It’s betting by deeply educated experts. How did their predictions go?

The “base rate” prediction is that Magnus Carlsen—the world #1 in all three major categories of chess, arguably the best player to have ever lived—should win, and indeed he was given the highest probability of winning before the tournament began, even up to round 4.

But the world doesn’t necessarily unfold around the base-rate. Nordirbek Abdusattorov—an 18-year-old junior player rated more than 100 Elo points lower than Carlsen (an enormous gap)—was given such a small sliver of a chance at the start that it’s hard to even read the number. He lead the tournament standings until the very last game. He also beat Carlsen in round 5.

And Giri was never given more than a 30% chance; usually 10% or less. He won the entire thing.

The same thing happened during the last FIFA World Cup. Argentina—the eventual winner—was never given much of a chance; even in the quarter-finals the bets were on France:

When we build strategies, we’re predicting the competitive market and which products will be winners. Chess and soccer games have a higher variance than companies, but analyzing sales funnels sometimes feels a bit like predicting Tata Steel. How do you build a strategy around this level of unpredictability?

Predicting product

Stewart Butterfield always wanted to make a game. So, in 2002, he did:

It failed, but it had some interesting features. Alpha testers liked that they could share game objects by dragging them into chat windows. So, the engineers created an online application for real-time chat with image-sharing.

Stewart Butterfield is the founder of Slack, so you might guess that this “better chat” idea became Slack, but no.

The chat application only worked in real-time—your pictures didn’t stick around when you closed the app. And this was fatal because it turns out people were interested in the sharing part more than the real-time part. So in yet another upheaval they rewrote the Flash application as a regular website, and Flickr was born. By 2008 it had become the largest photo-sharing site in the world with 3 billion photos and 5,000 more uploaded every minute.

Yahoo bought Flickr for $25M in 2005; Butterfield left three years later. Now that he had money in his pocket, he was free to go back and do what he always wanted to do, which was to make a game. So, in 2009, he did:

There’s a whole story about their dreams of a massively-multi-player experience, with APIs so others would build even more things on top of the game.

But it doesn’t matter, because once again it was a failure, and shut down. But once again there was a piece of the game that people really liked, and once again it was the chat system. Butterfield pivoted the company with a new mission, and a new name that belied the mission: SLACK, the Searchable Log of All Conversation and Knowledge.

Still not “chat,” but rather a “searchable log of knowledge.” It’s too hard, Butterfield insisted, for corporate denizens to search Google docs, and email, and chat sessions, and intranets, and knowledge bases, and support channels, and sales logs. Slack brings all that content together into a single omni-search, and thus solve a knowledge-management problem common to all companies.

Butterfield was certain that merely “building yet another chat system” was a bad strategy, whereas “transforming communications” was a good strategy:

We are unlikely to be able to sell “a group chat system” very well: there are just not enough people shopping for group chat.

That’s why what we’re selling is organizational transformation.
—Stewart Butterfield, Slack founder, in 2014

Once again, great theory, great mission, great strategy… and it didn’t work. Slack was, and is, yet another chat system.

Fortunately for him, Butterfield was also wrong that “just chat” wouldn’t sell very well. Slack was one of the fastest-growing companies in history:

What it is about people not wanting to build chat? WhatsApp has a parallel story. Initially it was just a way to post a public status message, so friends could see what everyone is up to, hence “WhatsApp” (like “What’s Up”). No one cared. Then the iPhone launched push-notifications, so they added a feature to alert you when a friend’s status changed. People started abusing this as a crude form of group-chat. So they added group-chat as a proper feature. The correct strategy was simple: SMS costs money for most people in the world, so a winning product strategy is “SMS, but free.” That strategy worked:

When we build strategies, we’re predicting how customers will receive our product, how it will solve problems or be delightful. And yet so often the idea we start with isn’t the idea that wins. Even once Slack became “Slack” and not “Glitch,” the strategy was still wrong. How can we ever be right?

The Solution

The Slack and WhatsApp stories of unpredictable success have at least two things in common:

1. They had a strategy—a philosophy of what would be entertaining or useful, a design sense of what would be delightful, brand new ideas for features and user interactions that people loved. They didn’t just throw random things at the wall.
2. They went where the customers took them—if not a game, then sharing images; if not another game, then better chat; if not corporate omni-search, then chat with APIs, if not status updates, then free SMS.

A strategy is required, even when it’s wrong.

A strategy gives you a direction. A direction creates something interesting. Something interesting might be used by zealous early fans, even if it ends up uncovering the answer rather than being the answer.

If Game Neverending wasn’t delightful enough to attract alpha testers, there would never have been a reboot into Flickr; if Glitch didn’t have 150,000 users at its peak, there wouldn’t be the idea for Slack; if Slack didn’t have enormous ambitions, it would not have received VC funding (for the same reason Butterfield himself gave for why “yet another chat” isn’t valuable).

A corollary: Making a decision and moving forward is often more effective than extended deliberation about the decision. Deliberation assumes we know how to reason about the future, but even experts aren’t good at that. Making decisions and gaining experience is how to find the right answers, if the organization is introspective enough to also face the truth when it turns out the original strategy is incorrect.

Dispense with the idea that there is One True Solution to the puzzles, and that the way to get there is to gather enough information. While “iterate fast, learn fast” is good advice, it still doesn’t mean you’re iterating towards the One True Solution. It means you’re taking a path towards something, which will turn out to be an integration between your own creation and the swirling reality outside of you.

More than any other time in history, mankind faces a crossroads. One path leads to despair and utter hopelessness. The other, to total extinction. Let us pray we have the wisdom to choose correctly.
—Woody Allen

The customer (behavior) is always (directionally) right.

There is a contradiction in these two points. Point (1) is to have a firm strategy, yet point (2) is to go wherever the customers are, even if that violates (1). If you just do “whatever customers want,” how is that a strategy?

The answer is: Customers are where you discover how to upgrade your strategy. Since you know your initial strategy is wrong, following customers’ lead is how you correct it.

When a customer loves one feature especially, or asks for adjacent features, those requests themselves are not your new strategy; the intent behind them might be. So “give me push notifications for my status updates” is a feature request, but the reason they want that is to circumvent SMS, and incorporating that idea is the strategy upgrade.

Perhaps the most interesting signal is when the customer abuses your product to accomplish something else. The desire for that “something else” is so great, they’re willing to use the wrong tool to get it done. Not only exemplified by WhatsApp users abusing “my status” to be “group chat,” but also in my own experience with Smart Bear:

Our first product (Code Historian) let you visualize the history of your codebase, which seemed cool to me but few people paid for it:

But people abused it to enable peer review. We were getting feature requests like: “Let me package up what’s on my screen and send it to someone else,” and “Let me write directly on a line of code and send it back.”

A new product was born (Code Collaborator) which within a few years represented 97% of the revenue of the company. “Abuse” is a strong, positive signal for customer demand, and worthy of a change in strategy.

So yes, the future is unpredictable. But that doesn’t absolve you from creating a strategy, and it certainly doesn’t mean you should accept that life is a random walk, with no way to bias results in your favor. In fact, sometimes strategies are correct right from the start, like it was with Google, LinkedIn, Amazon, Apple, and my latest company WP Engine¹.

¹
Although one reason WP Engine was correct at launch was because I used a system to discover what was important before building it.

You need a strategy, a fixation like Butterfield has for a certain type of game, a galvanizing reason for everyone to act together with a common purpose. And yet the strategy is always under suspicion, always updating, always reacting to unpredictable realities.

How, exactly, do you do this?

Strategies that defeat unpredictability

While it might not be possible to predict the future, it is possible for a strategy to side-step unpredictability through these mechanisms:

Build a moat

Covered in this companion article, a “moat” is a long-term, durable competitive advantage. A structural advantage that others cannot disrupt, lasting for years, is resilient to the volatility of your competitive market.

Have more than one way to succeed

If multiple things all have to go right for a plan to succeed, it probably won’t succeed. That sort of plan tacitly assumes we can predict the future of many different components; surely a bad bet.

This is why startups are difficult in the first place: We have to have an insight and build a compelling product and be able to reach target customers and do that cost-effectively and at a price customers will accept and be better than the competition for some segment and not lose a key team member early on and not have global economic failure…

A resilient plan is built of “or” not “and:” We could reach customers through social media or paid advertisement or influencer marketing or channel sales. A product that at least could be sold in all those ways is more likely to succeed than a product which can only be sold a single way.

Optionality defeats unpredictability:

• A product that is very low cost to create has many options for pricing, and therefore more likely to find effective pricing.
• A product in a large, growing market has many niches and personas and channels to potentially target, and therefore it’s more likely you’ll find some combination that works.
• Some indie developers build multiple small products, then pour their effort into whichever one happens to take off.

More examples are given in Nassim Taleb’s concept of Antifragility. In his words:

If you “have optionality,” you don’t have much need for what is commonly called intelligence, knowledge, insight, skills, and these complicated things that take place in our brain cells. For you don’t have to be right that often. All you need is the wisdom to not do unintelligent things to hurt yourself (some acts of omission) and recognize favorable outcomes when they occur.
—Nassim Taleb, Antifragile: Things That Gain from Disorder (2014)

Leverage your assets

“Strike where you are strong and the enemy is weak” said Sun Tzu 2,500 years ago, echoing into strategy-design from the SWOT of the 1970s to modern agile frameworks like the “Means” in Effectuation.

This is obvious but often forgotten by entrepreneurs who believe they have found lucrative opportunities with amazing product ideas, but which fall outside of their sphere of competency, where the enemy is strong and they are weak. Any venture is likely to fail for many—unpredictable—reasons; the very least you can do, is leverage your capabilities, knowledge, network, (professional) friends, and accumulated assets. That might mean building a very different kind of company or product or target market; in fact that’s exactly the point. When you pick the battle you are best-suited to win, you have a higher chance of winning regardless of the unpredictability of the world around you.

Intentionally reactive

If you cannot predict the future, one option is to spend no time whatsoever trying to plan. Just react to what you see now, and use your judgement and a long-term strategy to solve the immediate opportunities while advancing a long-term agenda.

We know, we know… iterate fast, learn fast, get to the right answer fast. We know that, we say that, but then we take four months to deliver one feature, or take 18 months before we’re “ready” to launch. Truly embracing and living the idea of constant delivery, constant feedback, constant learning loops, constant adjustment of hypothesis, quick decision-making, quick updating of prior decisions, no ego tied up with who was right or wrong about what, treating all doors as two-way doors, more than lip-service but encoded into the DNA of the organization, can overcome unpredictable barriers. There was no real prediction to begin with, just a constant stream of hypotheses, continuously updated, possibly using frameworks and processes specifically designed for this mindset².

²
e.g. building in public; the OODA loop; Iterative Hypothesis Testing; Continuous Discovery, Continuous Delivery (not to be confused with Continuous Deployment, in which deployment is automated but isn’t happening many times per day)

Hedged bets

If you buy 30-year life insurance and live for 60 more years, you will make far more money than the insurance costs, so you come out ahead. If you buy 30-year life insurance and die in 2 years, your family will receive far more money than the insurance costs, so you come out ahead. Thus “insurance” is a way to always come out ahead, without predicting the future.

The catch: You’ve reduced your maximum upside. In the 60-year case, you paid insurance premiums for 30 years while receiving no money in return. On balance, however, this “tax” is worth it, because you’re trading slightly less maximum upside for predictable, net-positive outcomes.

Strategies can also “buy insurance,” i.e. come out ahead either way, albeit with a tax. Examples:

• Multiple vendors for same service³. Shift workloads based on price and performance. (How would your price-negotiations go, if you were in this position of power?)
• Multiple brands (whether created or acquired). Each finds a different niche, some will fail, some will take off. Common with acquisitions of growing companies, or the “house of brands” business model, or the “holding company” investment vehicle.
• Multiple simultaneous solutions. To create iOS, Apple had the iPod team try to expand their OS to be more capable, while simultaneously having the Macintosh team try to reduce their OS to fit on a mobile device. One result was in-fighting, demonstrating that “cost” can be personal. But the result was that they selected the correct way forward for the iPhone.
• Redundant systems. City power, and a generator. Multiple servers in different geographic regions. Teams, not solo engineers. Lying dials in airplanes. Redundancy is a cost, but the result is more predictable operation. As the military saying goes, “Two is one, one is none.”
• Disrupt thyself. Create new products that disrupt your existing products. The quintessential example is Amazon launching the Kindle; Bezos instructed the newly-annointed Kindle leader—who was the leader over their book-selling business—that “Effective tomorrow, your job is to kill your old business with a Kindle.” Whether the future is in physical books, electronic books, or both, Amazon prospers.

³
Wrap 3rd-party APIs in a proxy layer. Use multiple public clouds. Process online transactions with more than one party. Keep your cash in more than one bank.

Extreme novelty

Zappos, Airbnb, Uber, SpaceX, Tesla, Bitcoin, OpenAI, all took enormous risk by creating a new category that everyone else thought was impossible, even after they heard the plan. On one hand, this is the definition of unpredictability. On the other hand, they avoided the unpredictability that comes from existing competition in existing markets. It doesn’t matter what’s happening at Amazon if you have free returns for 364 on shoes; it doesn’t matter how the hotel industry is shifting when you’re selling a different experience; it doesn’t matter what Ford and Toyota are doing with electric vehicles if you’re changing the fundamental technology without the constraints of existing supply lines, dealer distribution rules, and ties to the oil and gas industry.

If everything is unpredictable anyway, why not earn strategic advantage: No direct competition, exciting place for top-talent to join, immense upside.

That upside does need to be immense for the risk to be rational. If it is, this a strategy for trading into a better set of uncertainties.

Form coalitions

If you want to go fast, go alone. If you want to go far, go together
—African proverb

The more massive the object, the more it resists being moved by external forces.

Price-collusion⁴ removes the uncertainty of market pricing. Peace treaties prevent some wars. Open source projects attract more talent and advocates than any one organization could afford. Industry standards reduce the risk that core technology or protocols change, allowing members to build systems on top of those standard for decades, not having to “hedge.” Good things happen when we all agree to use HTTP, HTML, and SSL.

⁴
This is unethical and illegal in many countries; don’t do it. It is illegal because it works, which makes it a good example.

Expand the scope of prediction

Predicting exactly what will happen is folly, but mapping possible futures can be illuminating:

Plotting possibilities that you believe are improbable, helps you recognize that perhaps they’re not quite as improbable, helps you think of solutions that mitigate plausible challenges⁵, and help you recognize if an “improbable” thing is in fact happening.

⁵
The “pre-mortem” is an increasingly popular workshop for accomplishing this, in which you brainstorm answers to the following question: “It’s 12 months from now, and the project is a disaster. What went wrong?” The idea isn’t to solve everything you can imagine, but rather to pick a few things to intentionally mitigate, and intentionally leave the door open for mitigating other things if they come to pass.

Of course you’re still ignorant not only of probabilities, but of eventualities that you never predicted. Gaalen acknowledges this by further expanding the framework:

While this doesn’t mean you suddenly can predict the future, it might mean you’re paying more attention to what’s actually happening, allowing you to react faster, and thus to manage the unpredictability better.

Stay simple

The more tasks have to be broken-down, the more dependency arrows are drawn, the more intricate the analysis, the less believable the project estimate is. Complexity breeds unpredictability.

The inverse is that simplicity is predictable. While not strictly true, is it true that simple things can be more predictable. A simple product, with a simple value proposition, in a simple market, has fewer dependencies that require prediction, and thus are more likely to succeed.

Bet on what will not change

Made famous by Jeff Bezos, it is difficult to predict how the future will change, but it can be easy to predict the ways the future will not change. In Amazon’s case, he cites “low prices” and “fast delivery” as two of those things; in ten years, people will still want that. Therefore, Amazon can (and does) invest billions of dollars to achieve those results.

In the case of WhatsApp, consumers want to chat and don’t want to pay; that was true ten years ago, it’s still true today, and it’s a good bet that it will be true ten years from now. In the case of WP Engine, the company I founded 13 years ago, people wanted websites to be fast, stay fast even when they get a lot of traffic, and be secure; they still do, and I’m sure they still will in another 13 years.

And so the lessons are:

• Have a strategy, even though the world is unpredictable.
• Decide quickly → get customer reactions quickly → learn quickly → make new decisions quickly.
• Upgrade the strategy by following customer behavior.
• Product “abuse” is a strong signal for updating the strategy.
• The strategy must include building a moat or two.
• The strategy must create optionality in how to succeed (or avoid failure) so that single failures aren’t fatal.
• The strategy must leverage your existing strengths, building a product for a market for which you are already well-suited.
• Either keep it simple, form coalitions, or do something completely novel.
• Bet on things that won’t change, rather than predicting how things will change.
• Do your homework, but don’t stall in analysis paralysis. Map future possibilities, to mitigate possible things and to better notice when your assumptions turn out to be incorrect.

That’s how you win in an unpredictable world.

Printed from: A Smart Bear
https://longform.asmartbear.com/predict-the-future/
© 2007-2023 Jason Cohen
@asmartbear