The Past and Future of Product Management

A few months ago, I was asked to deliver a presentation for a gathering of newsroom product managers at the American Press Institute. This presentation encapsulates my feelings about the future of product management based on experience, observation, and (numerous) mistakes. It is, in many ways, my life’s work so far.

This presentation was written as annotated slides, so it is presented below as annotated slides. To briefly summarize:

• I believe that the future of product management rests upon our ability to embrace human complexity, in both the processes we implement to build products and the data we consult to understand customers.
• I believe that product management is a fundamentally supportive and facilitative role, not a “visionary” role. (No Steve Jobses need apply.)
• I believe that the distinction between “hard skills” and “soft skills” is largely counterproductive, and that the best product managers possess the connective skills needed to bridge diverse perspectives and roles within an organization.
• I believe that many of the traditional “profiles” of product managers are narrow and misguided, and there is enormous untapped potential for great product managers beyond “engineers who like design” or “UX designers who can code.”

If this seems affirming, intriguing, or flat-out wrong to you, please read on.

Today I’d like to talk about the future of product management by outlining four major shifts that I’ve observed over the past five years.

But before getting into those specific shifts, I want to address one other high-level change I’ve noticed. When I started working as a product manager, I heard this question a lot: “Why should I hire product managers?” In some cases, I would hear CTOs and technically-minded founders openly brag about the fact that they were able to ship software without hiring product managers at all. Product managers sometimes seemed like a necessary evil, a grudging concession to the world of corporate org charts and P&L statements.

Much of this came down to the mistaken belief that product managers were simply second-rate developers who couldn’t cut it in a purely technical role. You needed developers to get software shipped; product managers were at best a nice-to-have, at worst an impediment to the “real” work of writing and deploying code.

That mindset still exists in some organizations, but I find myself encountering it far less often. Instead, I am often asked: “how do I hire product managers?”

Part of the reason for this, I believe, is that many high-profile and high-tech products have publicly failed to live up to their expectations. The idea that shipping software — any software — is itself a holistic end goal is much harder to support now than it was five years ago. And as venture capital becomes more interested in companies that are revenue-focused and truly understand their market, there is an appreciable shift away from “just ship software” and towards “ship the right software.”

The problem, though, is that the skill sets needed to ship the right software are much harder to pin down. Hiring developers is by no means easy, but mastery of technical systems is ultimately easier to display and quantify than mastery of human systems (if the latter is even possible). As more companies acknowledge the importance of product management, there is also a growing sense of anxiety and confusion around what makes a “good” product manager and how to find them.

(Ken Norton’s insightful [and ahead of its time] piece “How to Hire a Product Manager” is a must-read here, and something I find myself turning to often.)

I believe that the answers to these questions have changed over time as well, and today I would like to share with you four shifts I have observed in how we think about “good” product management. The first shift I’d like to discuss is that from capital-A Agile to lowercase-a agile.

What do I mean by this? It’s worth remembering that in its original form, Agile was a manifesto — a statement of values. And it was a set of values that encouraged — well, agility. In its purest form, “Agile” was about respecting the complexity of individuals and the inevitability of change.

It is somewhat ironic, then, that this set of values was codified into a highly prescriptive and static process…

… which then became an ecosystem of increasingly specific and minutely differentiated processes and tools…

… each of which required its own comprehensive documentation and set of rules.

In a sense, though, this made hiring product managers much easier. You pick a process, you hire an expert in that process. You’re done.

… but this approach has severe limitations. Notably, it means that you are hiring people who possess a fixed set of knowledge and experience, not people who excel at adaptability and quick thinking — people who are lowercase-a “agile.”

Andy Hunt, one of the signers of the Agile Manifesto, put this incredibly well in his blog post “The Failure of Agile,” citing what he calls the “joy of rules”:

The only way for beginners to be effective is to follow simple, context-free rules; rules of the form: ”When this happens, do that.” And since agile methods conveniently provide some concrete practices to start with, new teams latch on to those, or part of those, and get stuck there.

So instead of looking up to the agile principles and the abstract ideas of the agile manifesto, folks get as far as the perceived iron rules of a set of practices, and no further. Agile methods ask practitioners to think, and frankly, that‘s a hard sell. It is far more comfortable to simply follow what rules are given and claim you’re “doing it by the book.” It’s easy, it’s safe from ridicule or recrimination; you won’t get fired for it…. You are ”doing agile” if anyone asks, and you can focus your energy on enforcing a small set of largely useless rules. Everyone feels better. Until it’s time to actually ship something.

… and this, I think, speaks to one of the more challenging changes happening in the world of product management. Companies are realizing that product management isn’t about choosing the right process — it’s about building a practice within your entire organization.

What do I mean by a “practice”? In a sense, I’m saying here that product management is similar to yoga or mindfulness, in that you can’t master it by reading a book of rules. (And, if my experiences with yoga and meditation are any indication, it is not something that comes easily or naturally to everybody.) One of the most common mistakes I see companies make is implementing a new “Agile” process or hiring a new product manager and declaring either or both a failure when things don’t improve immediately. I’ve seen companies switch their product management process five times only to realize that the problems are — to return to the Agile Manifesto — about people and interactions, not tools and processes. Changing tools and processes is relatively easy, but changing people and interactions takes time, discomfort and reflection.

In discussing product management as a practice, I find this quote from Jon Kabat-Zinn about the practice of meditation very instructive:

[People] want to meditate in order to relax, to experience a special state, to become a better person, to reduce some stress or pain, to break out of old habits and patterns, to become free or enlightened. All valid reasons to take up meditation practice, but all equally fraught with problems if you expect those things to happen just because now you are meditating. You’ll get caught up in wanting to have a “special experience” or in looking for signs of progress, and if you don’t feel something special pretty quickly, you may start to doubt the path you have chosen, or to wonder whether you are “doing it right.”

The second change I’d like to discuss is from “data-driven” product management to “customer-driven” product management.

Perhaps owing to the generalized anxiety around product management as a constellation of “soft skills,” I’ve heard an increasing demand for “data-driven” product managers. And, really, who doesn’t want to be “data-driven” in this day and age? In my career as a PM, I’ve found that one quick way to establish authority and legitimacy is to invoke the quantitative certitude of “data,” to spend a lot of time with my screen full of dashboards, to devote ample time to managing and evaluating analytics tools.

But the job of a product manager isn’t to understand analytics, it is to understand customers. Too often, the term “data-driven” leads to a mindset that values numbers above the people from whom those numbers are collected. If our goal is simply to drive specific charts up and to the right, then we will optimize for whatever drives those specific charts up and to the right, even if it comes at the expense of our customers and the reality of our products.

My friend Mike Dewar, who is a data scientist at the New York Times, wrote an excellent piece about this very thing, arguing that recommendation engines are not about maximizing metrics, but rather about designing experiences for people.

One of the most damaging and persistent myths of the “big data” era is that, by looking at numbers and dashboards, we can know people “better than they know themselves.” This mindset betrays not only fundamental misunderstandings about “data,” but also fundamental misunderstandings about humanity itself. The idea that we can understand people without listening to them is hubristic, narrow-minded and, frankly, sad. Talking to actual humans can be confusing, awkward, and even downright discouraging. But if we really want to understand customers, we need to accept and embrace that people are not fully predictable and quantifiable.

This cartoon from Datascope Analytics captures something very important that I think we are sometimes reluctant to admit: without a clear purpose, “data-driven” work is often busywork. Sometimes, it is busywork that actively distances us from our customers by oversimplifying their needs and behaviors. If we aren’t asking “why” — if we aren’t looking for the messy human reasons behind our “data-driven” decisions — we are not doing our job as product managers.

Relatedly, the third shift I’d like to discuss is the shift from “what” to “why.”

When I started working as a PM, I was very excited to “own” the product roadmap. I was going to be the idea guy, the visionary who walks in and transforms the company for the better. Simply put, I was going to tell everyone what to build.

This idea is closely related to the notion that a product manager is the “mini-CEO” for a product. When I started working as a PM, I really liked this idea. It made me feel important and powerful. Hell, sometimes I even started thinking about this guy.

To that end, I LOVED when folks on my team asked me “what should I be working on?” It made me feel like a true product mastermind, the keeper of roadmaps and success metrics, the guardian of the product’s future. I alone understood the company’s deepest goals and desires, I alone developed the brilliant ideas that would accomplish those goals, and I alone would tell my team which of those ideas to execute and when.

Now this idea dovetails with something else I’ve heard that I want to interrogate — and this is, frankly, a huge mistake I’ve made. There’s a widely held notion that developers and other people who are primarily responsible for execution need to be “spared” from business or subject matter decisions. This idea that developers just want to write code with headphones on all day and that if you try to tell them about business goals, user insights — why you’re doing what you’re doing — you are wasting their time and offending their special, singular nature.

Not only do I believe that this is false — I also believe that it is ultimately condescending to engineers. I don’t really believe that anybody simply wants to execute without knowing why they’re doing what they’re doing — and I certainly don’t believe that this mindset should be encouraged. And, frankly, almost every time I’ve seen a product manager or executive withhold information about “why,” it’s because they don’t really know why.

What I wish somebody had told me when I started working as a product manager is: “Don’t be a hero.” Self-styled “visionary” product managers are dangerous — which is why I advise against hiring PM candidates who fire off S**** J*** quotes during job interviews. Product management is fundamentally about making connections and creating understanding across different roles, not about telling people what to do. It’s a supportive role, not a visionary role. Being asked “what should we work on next” may have made me feel good temporarily, but it usually meant that I was not actually doing a very good job as a PM.

This was especially true when our priorities or circumstances changed, which they always did. Descoping a project — or, hell, even initially scoping a project from a technical perspective — is incredibly difficult if you don’t know why you’re building what you’re building. If my team didn’t know why they were building what they were building, they couldn’t make the technical decisions that best supported the project’s goals.

Eventually, I stopped trying to be a product “leader” and instead did my best to make sure that the company’s goals were clear and transparent, and that these goals were well understood by as many people as possible. It made me feel less important, but it dramatically improved the way my team worked together. When I’m working with companies to build out a prioritization process, I now ask myself: “are the company’s goals clear enough that the product team will be able to prioritize just as effectively when I’m not here?”

Finally, I want to talk about the shift from “hard skills” and “soft skills” to connective skills.

I’ve often been asked how to balance “hard skills” and “soft skills” when hiring a PM. Frankly, I think this very distinction is often what stops companies from identifying the best potential PMs, and what stops PMs from feeling confident in their work.

Megan Kierstead, who is a great writer about product management and and user research, tells this story which I think may resonate with many PMs who have sat through a terrible job interview. This is, to me, exemplary of what happens when you think in terms of “hard skills” and “soft skills.” The very language of “hard” vs. “soft” inherently privileges one over the other and, perhaps most damagingly, suggests that this is somehow a zero-sum game where one must come at the expense of the other.

In a lot of the literature about product management, there is this idea that PMs need to be “technical enough.” This is something I’ve seen work well in some contexts, and something I’ve seen backfire in others. At its best, this idea compels companies to hire product managers who are curious and interested enough in technology to effectively delegate decisions and ask good questions. (The technical bar for being able to do these things is, by the way, quite low.) At its worst, this idea compels companies to hire the candidates who are the most like engineers, not the candidates who will make the best product managers.

PM candidates who are exactly like engineers are often the most well-received by engineering teams during the job interview process. And why wouldn’t they be? They often have similar experiences, similar interests, and similar expertise. The developers have a lot to talk to them about. And the hiring manager, who is afraid of alienating the engineers by bringing on a “non-technical” product manager, breathes a sigh of relief.

Unfortunately, these candidates often wind up failing engineering teams for the very reason they initially charmed them. Early on in my career as a product manager, when I was still clinging to whatever technical expertise I had as a source of legitimacy, I often found myself “protecting” the work that my engineers found most technically interesting. But as soon as the team was held to account for the customer-facing impact of that work, I realized that I wasn’t actually protecting anything except my own ego.

The truly great technical PMs I’ve met know how to invite their engineering teams into the prioritization process in such a way that the process itself is both interesting and rewarding. They know how to empower an engineering team with the information to make technical decisions that will support the company’s business goals. In other words, the differentiating factor between success and failure is not being “technical enough,” but rather being adept at making connections between different sets of values and expertise.

So why is the idea of the “technical enough” product manager so ubiquitous? To me, it sits at the intersection of two concepts that both seemed pretty good when taken at face value: “engineering-driven culture” and “culture fit.” In theory, of course the highly specialized workers building a technical product should work in an environment that makes them feel productive and empowered. And of course those people should work with people they respect and admire.

In practice, though, these ideas are often manifest as the belief that engineers will only “respect” somebody who shares their own skill set. The funny thing is that when I encounter this belief, it usually comes from a management team that is concerned about keeping engineers happy, and not from engineers themselves.

The notion of “culture fit” is a huge part of why companies have such a hard time hiring good PMs. Because the role is so variable, because it involves so many of those messy “people skills,” there’s a certain sense of safety in hiring people who basically look and act exactly like your engineering team, or your analytics team, or whatever team they’re directly collaborating with.

But I’ve recently heard people replacing the notion of “culture fit” — which, frankly, is a hilariously conservative yardstick for self-styled “innovative” companies — with the idea of “culture add.” This means embracing the idea that bringing in diverse skills and perspectives will expand your organization’s knowledge and capabilities. (My friend Melanie Stern wrote a great piece about culture add from a recruiting perspective.) I would add that it is also a much more generous way of approaching the people who are already on your team, assuming that they will be curious about new perspectives, rather than fearful of those perspectives.

And this curiosity, I think, is key to understanding the values that a good product manager can add to an organization. I am much more excited about PM candidates who tell me what they learned about team dynamics from their improv group or political organizing work than I am about candidates who rattle off the finer distinctions between scrum and XP. Great PMs can make connections between seemingly disparate experiences, value systems and skill sets.

When people ask me how to get in the good graces of engineers, data scientists, and other highly specialized talents, my answer is not “learn how to code.” It is, without fail, “take a genuine interest in the work that they do.” It took me years to understand this, but I now believe that it is an incontrovertible truth of product management. Modeling and cultivating genuine curiosity is a huge part of your job.

When I consult with organizations that are looking to identify internal candidates for PM roles, I often ask them to draw a map of how information travels within their organization. Not the org chart, just an informal sketch of how people communicate with each other. Oftentimes, a few folks will emerge as central nodes, communicators, connectors. These people usually don’t fit the traditional profile of a PM: they are not always engineers with an interest in design, nor are they always designers who can write some code. Oftentimes they’re in sales, or in community, or in marketing. Oftentimes, they are in entirely “non-technical” roles. But almost every company and team has at least one person who naturally liases between different perspectives and departments. These are the people who understand lower-case “a” agility and, crucially, these are the people who have already proven their capability at these difficult but critical connective skills. These are the people who I believe are the next great PMs.

If you’ve made it this far — or even if you just scrolled immediately to the bottom — my sincere and heartfelt thanks for reading.

Enormous gratitude to the American Press Institute and Sarah Milstein for hosting this talk.