Babe Ruth and Feature Lists

Why Prioritized Feature Lists Can Be Poisonous

In 2007 I was a product manager on Google Docs. Docs was barely a year old, stemming from the acquisition of a marvelous startup called Writely. We were still nailing the basics. (On the FUD timeline, Microsoft had not yet called me “naive” and it would be three more years before they’d accuse us of being “up way past our bedtime.”)

We were popular with legions of early adopters and we saw notable enthusiasm in the educational market. Teachers loved the idea of sharing documents with their students without the hassle and cost of software licenses. Our academic customers were passionate, smart, and vocal, which meant we got constant high-quality feedback. I responded by keeping a voluminous feature request list that was managed, quite naturally, in a Google Spreadsheet.

I was careful to track the priority and difficulty of the requests so we could spend our engineering time accordingly. Every time we got feedback from the forums, or talked to a customer, we made sure to reflect their requests in the spreadsheet, adding a new row if necessary.

My list emphasized broad themes such as formatting, printing, import/export, sharing, and commenting. We pushed hard to make sure each three-week release touched on as many top themes as possible. I felt good when a sprint made progress against all of our top priorities. Our sales and support counterparts passed along feedback from customers that reinforced our prioritization.

In March of 2008, the Google Apps for Business team invited a bevy of academics to Google’s Mountain View campus to meet various product managers. I was invited to join twenty K-12 and university educators and administrators to chat about Docs and hear their feedback first-hand. I was ready to share my top feature priority list, to reassure them that their biggest requests were represented. And maybe bounce a few big, longer term ideas off them.

My list probably began like this:

Google Docs Feature Priorities

1. Formatting
2. Printing
3. Import/Export
4. Sharing
5. Commenting
…

I started at the top and described some of the exciting new formatting features that were being considered: styles, template galleries, and more fonts. Almost immediately, one of the participants reacted: “Oh this is bad, I thought that meant you were going to fix formatting.” Others jumped in: “Google needs to stop working on fun things and start focusing on basic functionality. Please don’t waste your time on the non-essential stuff on that list.” Eventually the whole room piled on, and it was painful. For them, formatting was a bug, not a feature, and it was so crummy nothing else mattered.

“The more you use it, the worse it gets.”

Why was it so atrocious? We were relying on the browser’s rich text surface, which used HTML as the underlying data format and caused browser compatibility nightmares. You’d bold a word and then be unable to un-bold it. Bulleted lists couldn’t be edited or deleted without screwing up the whole page or turning everything into a bullet. Centering a line would often center-align the entire document. Formatting bugs had been an annoyance for Googlers, but they were fatal for groups of students who needed to print a term paper to a teacher’s exacting specifications.

“I spent all night fixing a paper that was due.”

I wanted to make sure the group wasn’t overreacting so I tried an exercise I learned from a friend: I asked the group to pretend they each had one hundred dollars of Google’s money to spend. How would they stack formatting bugs against these other improvements? The outcome was incontrovertible: except for a few scattered dollars, they chose to spend all of their play money to fix basic formatting bugs. One woman said: “I spend one hundred dollars on formatting, then I take another hundred of my own money out of my own damn wallet and spend that on fixing formatting.”

What Did I Learn?

To us, “formatting” was a broad product theme to hack at indefinitely. We had big plans for shared style templates and additional fonts. But to our users, “formatting” was an urgent bug that needed to be fixed. Sure, collaboration and the cloud were our differentiators, but we were failing to meet a minimum quality bar.

We were right to manage our product development against themes, but we messed up by not setting the right expectation with our customers. We all agreed formatting was a top priority, we just never agreed on the details.

Our wish list approach also created false equivalence. There was a huge chasm between what #1 meant to us and what it meant to our users. For us, it was first amongst equals. To them it was a painful tumor overdue for removal.

Orders of magnitude separated #1 from the rest of the list. That urgency didn’t come through until we got a bunch of them in the room and let them vent.

When you ask your customers to tell you what’s important, give them a blank sheet. Don’t ask them to react to a list. Having a real conversation allows you to ask follow-up questions like “but if you had to choose, would you rather have A or B?” Use the time to deeply understand the problems they’re encountering.

If you do share a feature list with customers, keep it short and cap the length. Tacking low priority requests onto the end of a bloated list causes it to become unmanageable and dilutes the significance of the top themes. Be clear and descriptive, and separate bugs from features. Keep a separate unranked pile for requests that haven’t been prioritized yet or slotted into a theme. Make sure everyone understands the criteria for removing something from the list.

Baseball Players, Ranked by Hitting Ability

1. Babe Ruth
2. William Van Winkle Wolf
3. Shooty Babitt
4. Pickles Dilhoeffer
5. Stoney McGlynn

Epilogue

That customer summit was just the jolt we needed to prioritize basic formatting fundamentals. We knew it was going to be hard and it would take a long time. An amazing team of engineers would write our own editing surface and layout engine from scratch and toss the browser’s built-in editor to the wind. (By the time the rewrite shipped I’d moved to a different project so I can’t take any credit.) Docs became even more awesome and is now used by more than five million businesses and millions more consumers. That rewrite set the foundation for all of today’s amazing features.

Planets of the Solar System, Ranked by Habitability

1. Earth
2. Mars
3. Jupiter
4. Saturn
5. Venus

I recently became a partner at Google Ventures after almost seven years as a Google product manager. I work hands on with our portfolio companies at our Startup Lab. Follow me on Twitter @kennethn.

Other essays I’ve written about product management:

• How to hire a product manager
• How to work with software engineers
• Leading cross-functional teams