The Dirty Secret of High Performing Product Teams

The dirty secret of Silicon Valley is that most great product teams follow a system that resembles waterfall (gasp!) to launch new innovative features/products repeatably. The system starts with high conviction based on judgment, intuition, and instinct rather than relying on iterative customer feedback to build conviction over time.

The process of going from 0 to 1 with a first product is an innovation. It’s what allows the company to get off the ground. Sometimes, that original innovation is enough to carry a company from seed to IPO. But that is incredibly rare. What’s more common is that companies need to ship big bets over and over in order to create step changes that help them scale.

There are certain teams that have clearly done this well. Stripe is probably the best recent example. Consider their product launches from the past 4 years:

Stripe

• Atlas – Feb 2016 – Startup Incorporation

• Sigma – June 2017 – Custom reporting and analytics

• Radar – April 2018 – Fraud and risk

• Billing – April 2018 – Subscription payments

• Issuing – July 2018 – Issue and create credit and debit cards

• Terminal – September 2018 – Physical payments

• Corporate Card – September 2019 – Stripe credit card

• Capital – September 2019 – Business Financing

• Climate – October 2020 – Carbon Removal Credits

• Treasury – December 2020 – Banking as a service

In addition, this is something we executed at our respective companies Wealthfront and Credit Karma.

Credit Karma

• Mid 2017 – Email Re-platform – A complete re-platform of the largest engagement channel for the product to enable personalization, ML targeting and ranking, and higher capacity.

• Late 2017 – CK Tax – Launching a new product line, free tax prep, in a highly complicated vertical.

• 2018 – CK Home -Launching new Home products within CK.

• 2018 – CK UK – Launching a revamped product built specifically for the UK, which operates on a different type of credit system.

• 2019 – CK Savings – Launching a new product line of Savings.

Wealthfront

• Mid 2017 – 529 Plans

• Mid 2017 – Borrowing / Credit Line

• Early 2018 – Financial Planning

• Early 2018 – Risk Parity investment account

• Early 2019 – Savings Account

• Mid 2020 – Debit Card + Checking

• Late 2020 – Auto Pilot – Automated Savings

But this doesn’t just exist in fintech. Snap in consumer social, HubSpot and Drift in B2B SaaS, and more have been able to execute on this system. There are also companies that have not been able to do this, such as Evernote and Yahoo. So what is the difference?

The Religion of Agile or Lean

“Agile” or “Lean” — processes that emphasize short development timelines and small iterations — have been heavily emphasized in Silicon Valley for the past decade. While there is a time and place for those systems, they:

1. Do not enable shipping big bets repeatably.

2. Lead companies/individuals to be less willing to take big swings.

3. Consistently fail companies when they realize they need a step function change.

High-performing product teams don’t religiously subscribe to one tool for all problems. They recognize that through a company’s lifetime, different problems arise that each requires a different tool.

One of the most underrepresented tools in the tool belt right now is being able to build, orchestrate, and manage a system that enables teams to accept and manage the uncertainty of high risk, high reward bets. Not having this tool cripples both company growth and the growth of an individual’s product or engineering career. Let’s first explore why.

The Gradient of Execution

In Crossing The Canyon: From Product Manager to Product Leader, Fareed Mosavat (General Partner at Reforge, Ex Dir Product at Slack) and Casey Winters (CPO at Eventbrite) wrote that one major axis of career progression is the scope of the problem/solution that you are responsible for executing on:

In both Product and Engineering, you are expected to be able to execute on large and more complex things in a high-quality way. The typical gradient of execution:

1. Maintain + Execute An Enhancement → You start by owning quality execution on the enhancement on a small area of the product.

2. Maintain and Evolve a Feature → You progress to owning execution on an individual feature.

3. Maintain and Evolve a Group of Features – Then you are expected to be able to maintain and evolve a moderately-sized part of the product stack. The scope and risk is contained, but you work on something of elevated status.

4. Launch Something New End to End → Then you are expected to be able to go from maintaining/evolving to launching something new end to end with higher complexity and scope. At this point, the complexity, risk, and scope of the things you are responsible for launching continue to grow.

Being able to manage the end to end launch of broader, larger, more complex, higher risk, higher reward initiatives is critical to progressing in Product and Engineering.

“Leading something that occurs in your team across a cross-functional, cross-team effort that requires dependencies, timelines, project plans, and all these new and complicated pieces can be career-defining projects.” – Matt Greenberg – CTO @ Reforge, Former VP Eng at Credit Karma

But many get stuck around steps 3 and 4.

There are many reasons for this, but two big ones are:

1. The Grey Area Grows

   The larger scale the project the more ambiguity there is in strategic decisions and tactical tradeoffs. Engineers especially are trained to find the “right” answer. Leaders get bogged down in trying to make the best decision but increase the time you spend on the wrong decisions. You’ve probably experienced a discussion similar to “This is 3% better than that.” But the real discussion is, does this matter? As you progress in your career, focusing on ways to get to fast, efficient, and reasonable solutions ends up being more important than coming to the “right” answer. This is a leap that many struggle with.

2. Multiple Teams, Multiple Sequences

   Larger projects involve figuring out how to execute across multiple teams that you do not directly own in multiple sequences. Your influence abilities need to move from explicit to implicit. But a lot of people struggle with “What do I do when I can’t tell them what to do?” They want everyone to know they are in charge and to be blessed to make every decision. But they fail to see the problem as them and it leads them to build teams who see their efforts as “us against the world” with major efforts that end in frustration and animosity.

How Continuous Big Bets Drive Company Growth

Shipping big bets aren’t just about career progression, but also company growth. Teams need to realize that every dimension of their business will inevitably face headwinds.

1. CAC Going Up – CAC eventually goes up due to increased channel saturation, competition, or lower intent audiences.

2. Available Target Market Decreasing – As you grow into your target market, the remaining available target market shrinks.

3. The Adjacent User – As you grow, your product brings in lower-intent, more casual users that can lead to lower retention and monetization.

4. Competition Increasing – Copying a competitor’s feature is standard business practice these days. The only way to win is to be the one that is leading and pushing the line forward.

5. Law of Large Numbers – In order to maintain a healthy % growth, you need bigger and bigger absolute gains.

We faced all of these at Credit Karma, but I’ll give an example of this last one. At one point the core business figured out how to grow +10 million users per year. That growth was amazing and drove revenue growth for the whole company except overtime each additional 10 million meant a smaller percentage of actual growth. As acquisition became a lever that wasn’t enough to fuel the rocket ship, the attention shifted to engagement. We made amazing gains in engagement but finally the company was facing headwinds that could only be solved with meaningful new innovation.

As companies scale, they need standardized processes on how to build and launch new bets in order to drive additional growth. This is commonly framed in the context of the S curve. As we discuss in Advanced Growth Strategy and Product Strategy, there are two types of these bets.

1. Raising The Existing Ceiling – Shipping new innovative features within your existing use cases can increase the ceiling on your existing product.

2. Creating A New Ceiling – At some point you can’t find the gains needed just by raising the ceiling on your existing product. You need to go find entirely new adjacent products and use cases to create new ceilings.

These two methods point to the most common breaking points where defining this system comes into play:

1. Series B+ – Many companies around the Series B stage finally get the financial and human capital to pull off multiple major initiatives at the same time, but they fail to develop a system that allows them to do so reliably. We experienced this at Wealthfront, where between 2014 and 2017 we failed to make the transition, and ended up delivering a lot of low-quality, inconsistent, and undifferentiated products. All we did was burn our investors’ capital.

2. Series D and Beyond – In late-stage private companies, there typically comes a time where you need to start expanding your portfolio of products. Brian Balfour talks about this at HubSpot:

   “I joined HubSpot about a year before the IPO. At the time, we had one product, the Marketing Hub. I remember a meeting where Brian Halligan (CEO) and JD Sherman (COO) put a slide up that showed what happened to the growth rate of the past 5 Boston companies that went public. They all flatlined right after the IPO. They made it a goal that the same thing would not happen, and that gave us the “why “to plant multiple new seeds that ultimately became the HubSpot CRM and HubSpot Sales Hub products.”

The Messy Path From Strategic Idea To Outcome

To fuel career progression and company growth, we need a defined system to go from strategic idea to outcome. In our head, we often envision the path from strategic idea to outcome as something like this:

Scaling product delivery is challenging because execution is very rarely a clean straight line. In practice, so many things can derail you on the path from strategic idea to outcome:

• Uncertainty muddies what the path forward will entail – it’s unclear how much time it will take to build, what you’ll need to build, or what other dependencies you’ll face.

• Risks manifest that derail your efforts, such as facing a customer support constraint or losing access to a dependency team.

• Misalignment occurs that can prevent you from moving forward.

• Cannibalization is created because actions that you take in one area of the product have effects on other areas and can detract from one another.

So in reality, going from strategic idea to outcome looks more like this:

A lot of teams will explain these things away, saying things like “there’s no way we could’ve known this would happen.” The best leaders and companies out there have a system to avoid enough of these “unforeseen” blockers or deal with them eloquently as they come up in order to deliver outcomes for their customers/users and the business time after time.

Recognizing The Signals Of Unscalable Product Execution

So how do you know if you don’t have a scalable system? It can be hard to recognize what is a normal part of taking big bets, and when it is a problem with how you are managing these initiatives. Some of the signals that you need to level up on delivering the product are:

1. Late-stage blow-ups.

2. Inconsistent tradeoff decisions.

3. Growth is slowing despite rapid iterative development.

4. Eng + Product grow in size, but you aren’t shipping larger higher-quality bets.

5. Customer feedback changes from wanting more to complaints.

Let’s dive a bit deeper into each of these.

Late Stage Blow-Ups

Here is a common scenario: There is a big idea the team is excited about. You spend months building something, but then in the homestretch right before you are ready to launch, an executive or cofounder drops in from above and says “this isn’t right, we can’t launch it.” Being part of a late-stage blow-up can be one of the most demoralizing and worst experiences as a Product or Engineering leader.

This happens because, between idea and outcome, the system (or lack thereof) creates communication and expectation gaps between what executives have in their heads and the reality of what is built along the way.

Engineering + Product Grows, But What You Ship Doesn’t

When teams enter into hyper-growth, it often becomes about hiring. You have a team of 50 engineers, and you’ve been asked to double the size of the eng team each year for the next 2 years. Suddenly you have a 200 person team, but the output doesn’t feel like it has 4x’d with the team size. A task you were sure the team could have completed in a month ends up taking four months, even though the team working on it is bigger.

Scaling a team has many facets: infrastructure, recruiting, individual development, org structure etc. Even if you do all of them right, systems for many teams to work together on huge projects often break the mold and break the process. You need a model for getting big things done that doesn’t require the entire company to have to work on it as a top-down mandate or create a drag on the work of every team. Or else you see situations where your team grows four times as big but output stays the same.

Endless Tradeoff Debates

The path from strategic idea to outcome is messy. There will always be many forks in the road. But oftentimes, when teams hit a fork in the road, there is no explicit belief or manifesto about what is most important. Delivering on time? Feature richness of the product? Quality of the experience?

This devolves into an endless debate, or worse, being inconsistent throughout the process and optimizing in different directions. Part of good product strategy and execution is knowing which one of the tradeoff items is most important, so you can efficiently navigate through the forks in the road to a quality outcome.

Growth Slows Despite Rapid Iterative Development

The team is shipping a higher frequency of things and running more experiments, but growth continues to slow. This can mean a few different things. One, they aren’t experimenting in the right area. Two, they aren’t learning from experiments. Or three, it isn’t the time or place for experimentation entirely. The natural tendency is to “dig in” and go with one of the first two reasons, but it is worth stepping back and evaluating whether or not the solution is that you just need to ship bigger bets.

A Shift In Customer Feedback

When the team isn’t shipping bigger bets frequently enough, this starts to show up in customer feedback. The conversation shifts from people reaching out and asking for more, to people complaining about things that are broken. Customers take notice when you are constantly shipping big things; it changes their mindset and mentality.

The Root Causes

When Product and Engineering Leaders come across the above signals, it is important to recognize them as symptoms of a deeper root cause. A key mistake is to use band-aids rather than fixing the root of their execution problems. There are four common root causes:

No System

Some leaders have no standardized system to deliver large-scale initiatives. Each product and engineering team will unintentionally build products through different methods, leading to inconsistent delivery timelines and product quality.

How does this happen? Because we’ve all been trained that Waterfall is horrible and Agile is great. As a result, very few companies are willing to put a system in place that isn’t Agile-informed, due to the backlash it could create. This leads the team to get stuck in small, iterative bets without the balance of shipping big bets.

Applying The Wrong System

Applying the wrong system can be just as troublesome. Going from strategic idea to outcome is solving a series of problems. Different tools solve different problems. Applying the wrong tool to the wrong problem can manifest in all the issues described above. This most commonly comes in two forms. One, applying iterative systems with low process rigidity (like Agile) to large-scale work. And two, forcing a system from one company to the next, such as applying the Amazon-style press release to a culture that doesn’t read. When the root cause is applying the wrong system, you often see overly complex models put in place at the wrong time like scaled agile team of teams or Spotify’s matrixed organization applied at small-scale companies.

“We have the wrong people.”

Companies commonly believe that it isn’t a system problem but it is a people problem. They don’t have the right type of people to execute this type of work. But you can get the “right” type of people (whatever that means), and if you plug them into the wrong system, the outcome is going to be exactly the same. Teams need to recognize that there is no unicorn that is going to save them. Anchoring on the belief that it’s a team problem is almost always the signal that the root cause is a leadership issue. Leadership needs to give the teams a system that they can execute.

Over-Relying On “Superstars”

A lot of times teams have their “10X ninja superstar.” The one or two people that the company trusts to rely on their intuition, take on big projects and deliver outcomes on time. The first trap is to keep feeding this person more big projects. However, superstar resources are an unreliable way to scale. Superstars can only take on so many projects and have a limited influence on the broader execution system. Over-reliance on superstars, weakens the rest of the team.

This then leads teams to try and find more “superstars.” This conversation commonly takes the form of “We need to hire another X.” But there are very few leaders with this kind of ability out of the box. The goal shouldn’t be to try and hire more of the superstar but to deconstruct and institutionalize the things that make them a superstar.

The Keys To Shipping Products At Scale

Build A Specific System For Large Product Bets

All product problems are not created equal. So why would we use one system to solve all types of product problems? In “Product Work Beyond Product Market Fit,” Casey Winters (Reforge Partner, CPO at Eventbrite) and Fareed Mosavat (Reforge GP, Former Slack) talked about how there are 4 different types of product problems, and treating them equally creates many problems:

“The life of any company starts with going from zero to one on initial product-market fit. It is a requirement to reach before any other types of product work become a priority. But after product-market fit, there are four categories of product problems.

• Feature Work – Creating and capturing value by extending a product’s functionality and market into incremental and adjacent areas.

• Growth Work – Creating and capturing value by accelerating adoption and usage by the existing market.

• Scaling Work – Focusing on bottlenecks to ensure the team can continue to move forward and take on new levels of feature, growth, and product-market fit expansion work.

• Product-Market Fit Expansion – Increasing the ceiling on product-market fit in a non-incremental way by expanding into an adjacent market, adjacent product, or both.

One of the most common conflicts I’ve seen is when product leaders try to apply a single development process, measure of success, and strategy to all product work. For instance, while some of the steps can be similar, growth and feature work are fundamentally different and a lot of energy is wasted trying to make them all fit into the same process, success metric, and approach.”

If you want to ship big innovative bets at scale, you cannot use the same execution system that teams use for more iterative work. You need to design a system and train the teams on how to use and when to use it. Teams should know which type of product problem they are working on, which tools/systems align to those problems, and where everyone is at in their respective processes.

Manage The Complexity, Don’t Try To Remove It

When creating and managing the system, the common tendency is try and remove complexity entirely. But this is impossible. There will always be a level of uncertainty involved in executing innovative and uncertain strategic ideas. Creating product-changing wins involves taking higher-risk, longer-term bets. Instead, you need to accept the uncertainty by building processes that mitigate this complexity, increase conviction, and reduce risk over time.

Some of the keys to managing complexity are:

1. Agile Inspiration, Not Religion

   The right system takes inspiration from two core agile principles but does not throw waterfall away in the process. Specifically, short timelines to workable prototypes and acceptance of new insights during development.

   Teams need to know how to get to workable prototypes in short time frames (weeks, not months). If you don’t, this creates a negative flywheel where larger bets feel riskier and riskier due to longer windows of validation, and as a result, fewer bets are taken. You use the prototype to build conviction around the big bet, then step back and build a more structured plan.

   There also needs to be a deliberate focus on finding new insights during development and having a way to accept and incorporate these new learnings. Without this acceptance, teams take an “I’m just going to shut my eyes and hope for the best” approach.

2. Having A Tolerance For Misfires

   Sometimes the price of admission for being a company that’s oriented around repeated big bets is you get slapped on the wrist, make a mistake, or have misfires. But that needs to be OK. Robinhood had a big misfire about a year ago with an innovative savings product. They eventually had to pause the delivery of that product until they resolved some of the regulatory questions and concerns. Robinhood eventually resolved those, and then six months later were able to bring the product to market and still knock it out of the park.

3. Deliberate Tradeoff Decision Framework

   You can’t foresee everything at the beginning of the execution process. It is inevitable that something will come up that forces a tradeoff decision between delivery time, complexity or feature richness of the product, and quality of the experience. These are some of the hairiest conversations, and everyone needs to have a structured and deliberate approach to the decision. In a large scale effort they come up dozens of times and consistency in decision making leads to consistency in quality of the outcome.

4. Codified Check Points With Specific Purposes

   Part of having a defined system is to have codified checkpoints throughout the journey, each with their own specific purpose. A common method is to have “Product Reviews.” But teams fall into two traps. One, using the same Product Review for all checkpoints in the process when you actually need different formats. Two, teams know that they should do Product Reviews, but have never been trained how to do them extremely well. Codified checkpoints help you hit the goldilocks test and are at the right layer of detail at the right points in the lifecycle of product development.

Communicating The System

When implementing a system, the struggle begins with how it is communicated. This often manifests as execs saying “We need XYX process to make better products.” The engineering and product teams then think, “Great, the execs just want to watch everything we do.” It creates a feeling of distrust from the beginning.

The “why” behind the process is what matters and should drive the narrative. I experienced this at Wealthfront:

“When I took over product at Wealthfront, I spoke with all of the PM/Design/Engineering leads and asked them what they hated about how we built products. From that feedback, I was able to create a list of the 5 most common ways in which the product process had been broken. I validated those 5 things with the leads. I then proposed we create more formal steps of a process to prevent us from making the same mistakes. Finally, I engaged the leads in creating the method so they felt bought into it.” – Andy Johns

Know and Customize To Your Superpower

The system that you design for innovative product delivery within your company needs to be tailored to your specific business, to the customer you’re serving, and to the market that you’re competing in because not all companies are the same.

The great companies are those that understand that each step in the process is an atomic unit that they can mold and adapt to suit their exact needs. They eventually pick one part of the process that they turn into their “superpower.” Common examples of this are Amazon’s Press Release or Airbnb’s 7 Star Review.

Scaling Product Delivery

Common questions at this point might be: Where do I get started? What does great vs. good vs. bad look like? What are the atomic units of a good system? I know what to do, but how do I do it? What if the broader company culture is very risk-averse? And a lot more.

Learning how to build, manage, and execute these processes as a leader is an incredibly deep topic, so we created an entire 6-week program on it: Scaling Product Delivery.

We go through each stage of the execution process and discuss not just what to do but how to do it.

• Week One: Building Conviction

• Week Two: Scoping A Dynamic Plan

• Week Three: Implementing Core Team Reviews

• Week Four: Developing A Launch Strategy

• Week Five: Application Part One – We spend a week applying pieces of what you learn and getting peer feedback.

• Week Six: Application Part Two – We spend a week applying pieces of what you learn and getting peer feedback.

At each stage, we cover how Product and Engineering Leaders:

• Get perspective of the other team: Understand constraints, challenges, and priorities unique to each function so that you can build the empathy needed to execute cohesively.

• Make Tradeoff Decisions: Every stage involves tradeoff decisions. We go deep on how to effectively estimate and manage trade-offs in scope, time and resources, heuristics.

• Incorporate Principles of Scaling: How to incorporate the different types of scaling, such as tech scaling and process scaling.

If you are an Engineering Leader (Senior/Staff Engineer, Tech Lead, Eng Manager/Director, etc) or Product Leader (Product Lead, Group PM, Dir, etc), we’d love to have you join us as a member and in the next cohort, and even better to bring your counterpart. This program is specifically designed to improve the most important relationship and model in the product development lifecycle – product and engineering.

Accelerate your development, expand your abilities, and become the leader you are meant to be.