How to avoid building products that fail

It’s all about needs.

“If I had asked people what they wanted, they would have said faster horses.” Far too often, we hear those words (supposedly spoken by Henry Ford) as a way to justify rushing headlong into executing a so-called innovation before the idea is tested with users. It’s worth noting that not only did Henry Ford probably never speak those words, it also turns out that kind of thinking resulted in “a catastrophic loss of market share from which [Ford] never recovered.”

The lesson we should take from this story is that it’s extremely dangerous to execute ideas without first identifying and testing assumptions about the value of those ideas. We shouldn’t jump to a solution before we understand the problem. And that’s what this post is about.

When it comes to building products, the starting point is — always—needs. Not what we assume would be cool, but what users or the business need to be successful. Different inputs into this process include the following, which we’ll discuss in detail in this post:

• User needs. We must have a good understanding of the market, the company’s customers (existing and potential), and their behaviors and attitudes. We should never be caught off guard by questions about the product’s target audience.
• Business needs. The “putting users first” mantra too often neglects the fact that a product exists to make money. Having revenue goals is not an excuse for bad design, though.
• Technical needs. Development needs get ignored much of the time in favor of the more tangible front-end and business requirements. Developers know the limitations of the product; they know what needs to be fixed, and they know what technical debt needs to be paid.

One of the biggest mistakes we can make in product development is jumping to execution before an appropriate planning cycle has been completed, so we need to give planning the attention it deserves. Let’s start with gathering user needs.

User Needs

The first thing we need to clarify is the difference between needs and features. We often make the mistake of equating product features with user needs. If you’ve ever used a household appliance you’ll know that this isn’t the case. Have you ever used more than one or two of the preset cycles on your washing machine? And how many different ways do you need to toast your bread? The evolution of household appliances is a perfect example of what happens when features are equated with value. We don’t need more ways to wash our clothes. We might need faster or quieter ways, sure. But as we know, more isn’t necessarily better. And that’s when users sometimes take matters into their own hands.

When the first reviews and usage statistics for Facebook Home started appearing, John Gruber used a phrase that stuck with me: “It’s a well-designed implementation of an idea no one wants.” Hyperbole aside, this is what happens when features (cover feed, friends filling the screen, chat heads, app launcher…) are mistaken for user needs (why would people want to replace their phone’s operating system with an app?). The distinction between features and needs is important, and sometimes difficult to spot. That’s where user research comes in.

Research methods for gathering user needs are powerful because they rely more on observation and deduction than gathering answers to a bunch of predetermined questions. But before we get into the different methods we can use to make better products, we need to take a little detour to define some basic research terms.

First, we need to distinguish between quantitative research and qualitative research. With quantitative approaches, data tends to be collected indirectly from respondents, through methods like surveys and web analytics. Quantitative research allows you to understand what is happening, or how much of it is happening. With qualitative approaches, data is collected directly from participants in the form of interviews or usability tests. Qualitative research helps you understand how or why certain behaviors occur.

We also need to make a distinction between market research and user research. Both are important, but they serve different purposes. Market research seeks to understand the needs of a market in general. It is concerned with things like brand equity and market positioning. Attitudinal surveys and focus groups are the bread-and-butter tools for market researchers. They are tasked to figure out how to position a product in the market. Surveys and focus groups are very useful to understand market trends and needs, but they won’t help you very much when it comes to the design of your product.

User research, on the other hand, focuses on users’ interactions with a product. It is concerned with how people interact with technology, and what we can learn from their wants, needs, and frustrations. Those are the methods we’ll focus on in this section.

So, with those definitions under our belts, let’s look at some of the most common user research methods available to us. We can generally classify methods in three different buckets.

1. Exploratory Research

Exploratory research is most useful when the goal is to discover the most important (and often unmet) needs that users have with the products and services around them. This includes methods like contextual inquiries (also called ethnographic research or field visits), participatory design sessions, and concept testing. The goal here is to find out where there are gaps in the way existing products solve users’ problems. New product or feature ideas often develop out of these sessions.

Make no mistake, this isn’t about asking people if they want faster horses — it’s about observing people and finding out that they want to get where they need to be much faster than they’re currently able to.

For example, we used to do a lot of ethnography with eBay sellers all over the world. By going into people’s homes and seeing how they managed their sales, we uncovered a major issue that web analytics or surveys would never be able to tell us about. Sellers all manage their sales in different ways, ranging from sticky notes stuck all around their monitors, to Excel spreadsheets with complicated formulas and pivot tables. Sellers were forced to make up their own process for something eBay should be helping them with: how to track sales progress, and learn from that. Through ethnography we uncovered an unmet user need that can be met with a variety of features on the site. But the need is the starting point.

2. Design Research

Design research helps to develop and refine product ideas that come out of the user needs analysis. Methods include traditional usability testing, RITE testing (rapid iterative testing and evaluation), and even quantitative methods like eye tracking. This class of research helps us during the design process to create better products for the problems we’re trying to solve for users. For example, we can build interactive prototypes and bring people into a usability lab, give them tasks to complete on the prototype, and uncover usability issues before we start the (expensive) development cycle. Since these are usually in-depth one-to-one interviews, it’s also a great opportunity to get more insight on how well certain features meet those customer needs we identified during exploratory research.

3. Assessment Research

Assessment research helps us figure out if the changes we’ve made really improve the product, or if we’re just spinning our wheels for nothing. This class of research is often overlooked, but it’s a crucial part of the product development cycle. Methods include surveys and web analytics to gives us a view of how our products perform over time, not just in terms of hard conversions, but also in terms of the attitudes of our users. These methods are most useful when combined with further design research to understand why we’re seeing the changes we see. For example, form analytics can tell us where people abandon a form. Once we’ve made usability improvements to the form, it’s important to assess if those changes made a difference to completion rates. Without assessment research we won’t know if we’re going in the right direction.

Business Needs

The web is littered with the remains of products that did a great job of fulfilling user needs, but never figured out a way to make money and become sustainable businesses. Over the past few years we’ve seen several beloved services on the web shut down because of a lack of revenue. Editorially was a fantastic collaborative writing and editing tool whose creators eventually realized that “Even if all of our users paid up, it wouldn’t be enough.”

A few months before that, the photo startup Everpix shut its doors, in part because it couldn’t afford the cloud storage bills. This happened despite having thousands of paying users on the platform. The founders later admitted that even though they were able to make a product that people genuinely loved, they spent too much time working on that product, and not enough time on growth and distribution.

These stories are complex and there are never easy answers. Yet it’s a common approach on the web to focus on getting as many users as possible as quickly as possible, and then figuring out how to make money out of them later. Call me old school if you must, but that’s just no way to build a business.

I’m not saying a new product needs to be profitable from day one (although that would be nice, of course), but there at least needs to be a plan — a few possible revenue streams — that will eventually lead to a sustainable business.

So where do these revenue stream ideas come from? Well, in many cases they come from customers; the research methods discussed above can also be used to figure out what people would be willing to pay for — and how much. But there are also several internal teams that spend most of their time thinking about business needs, and the we need to form strong allies with these teams. This includes the business development team, the sales and marketing teams, and the engineering team (yes, the engineering team — no one knows the product better than they do).

When it comes to growing the business, it’s useful to split activities into two buckets: eliminating bad revenue streams, and pursuing good revenue streams.

Eliminating Bad Revenue Streams

The Greek tragedian Sophocles once wrote, “Profit is sweet, even if it comes from deception.” We have to be wary of our own frailties when it comes to making money.

Deceiving people to make a quick buck might seem like a good idea at the time, but it is a short-term strategy that is bound to backfire — not to mention that it’s unethical.

In interface design we refer to deceptive techniques as dark patterns. A dark pattern is a type of interface that is specifically designed to trick people into buying stuff they don’t want. There are many examples, and the website http://darkpatterns.org/ provides a comprehensive list, but these techniques include examples like these:

• Some iOS apps for kids, like Talking Tom Cat, put random pop-ups on the screen at all points in the game to trick kids into making in-app purchases.
• On login, PayPal often shows a full-screen ad with only a small link at the top-right to close the ad and continue to your account section.
• Zynga’s game FarmVille was “engineered with one goal in mind: to coerce users into tending their virtual plots of land for as long as possible.”
• Ryanair buries the option to opt out of travel insurance in an unrelated dropdown menu so most people don’t realize they’re buying the insurance.

It might seem obvious to point out that some revenue streams are not ethical, and therefore not worth pursuing. The problem is that, very often, these methods work — they unfortunately do make money (at least in the short term). But they also have long-term consequences that are rarely considered. Once users figure out what’s going on and start complaining, these underhanded methods affect companies directly in the form of increased support costs and major reputation damage. Ryanair has become a poster child for dark patterns because of its insurance tactics. That’s not a good position to be in.

The thing is, very few people start their days thinking, “I wonder how I could deceive people today?” Instead, dark patterns and deceptive methods sneak up on us as potentially decent ideas that degenerate little by little until they become dark patterns.

We shouldn’t spend much time on this, except to say: watch out. Don’t fall into the dark pattern trap. An easy, obvious, but rarely applied rule of thumb is to ask of any potential revenue opportunity: “Would I be OK with it if a product asked me to do or pay for this?” If the answer is no, walk away. There are better paths out there. It might be more difficult to find those paths, but it’s worth sacrificing short-term success for long-term loyalty from customers. And besides, you’ll sleep better at night.

Sometimes a revenue stream starts out as a good idea, but a change in the external environment turns it into something undesirable. The problem is that by then it might already be a large source of revenue, which places the business in quite a predicament.

One such example is photos in search results on eBay. Back in 1995 when eBay was founded, storage was expensive. So it made sense to charge users a nominal amount to upload a photo to their listings. Fast forward a decade to 2005, and not only was storage cheap, but the idea of charging users to add photos to their listings seemed ludicrous. The problem was that by then, charging for photos was a significant revenue stream, so it was not an easy decision to make photos free.

Our user experience team partnered with the analytics team to reveal that showing photos in search results by default not only increased sales, but also had a positive impact on ratings of search results organization and helpfulness. It took a while, but eventually eBay made the brave decision to shut off that bad revenue stream and make photos free (for up to eight photos), and it never looked back.

When it comes to dealing with these accidental bad revenue streams, the best course of action is to conduct research to understand needs and motivations, coupled with A/B testing to get an accurate measure of the effect it would have on good revenue when the bad revenue stream is cut off.

Pursuing Good Revenue Streams

Good revenue streams can come from many different sources. Consumers are willing to pay for things as long as the value is immediately clear to them. And the entire product management process is built around finding that value first, and then building a product and business around it, as opposed to making something and then scrambling to find the value. So, user needs research is always the first place to look for how the product can make money.

Consider the slightly offbeat example of Iron Maiden (yes, really). The band has spent a significant amount of time touring in South America, and it’s turned out to be a great strategy. They usually play sold out, highly profitable shows in the region. Why has the band been so successful there? If you ask music analytics company Musicmetric, they’ll tell you it’s related to an unlikely phenomenon: piracy. The company discovered a surge of BitTorrent traffic related to Iron Maiden’s music in South America in recent years, particularly in Brazil. By going where people were already fans of the music the band was able, as Musicmetric CEO Gregory Mead put it, to “[be] rather successful in turning free file-sharing into fee-paying fans.” Understanding where your users are already highly invested in your product is the best way to figure out what value they would be willing to pay for.

Once there are existing revenue streams, there are several standard growth strategies, such as expanding to new regions, establishing new channels, appealing to a broader market, and building new products for an existing market. But there is one strategy in particular that I’d like to focus on, because it’s a comprehensive approach to long-term business success. It’s a strategy formalized by Brandon Schauer at Adaptive Path, and it’s called the long wow. To quote from Brandon’s article about it:

The Long Wow is a means to achieving long-term customer loyalty through systematically impressing your customers again and again. Going a step beyond just measuring loyalty, the Long Wow is an experience-centric approach to fostering and creating it.

The long wow is built on a four-step process:

1. Know your platform for delivery. Identify the ways you can combine different ways to engage with customers, both online and offline.
2. Tackle a wide area of unmet customer needs. Based on your user needs research, identify an area where there is a huge need that is not met by your product, or any other products out there.
3. Create and evolve your repeatable process. Combine the company’s existing strengths with new ideas to meet unmet needs to come up with ways to delight users over and over.
4. Plan and stage wow experiences. Develop your ideas over time, and introduce new and better experiences consistently along the product development life cycle.

And then, repeat as necessary to make sure the long wow isn’t just a one-time thing. This is an excellent way to identify good revenue streams in your product, and ensure that you continue to provide the value needed to create loyal customers who keep giving you money.

Technical Needs

The first thing we need to be clear about when it comes to technical needs is that, just as in finance, there is a huge difference between assets and debt. Technical assets are things like the underlying technologies your product is built on, backoffice systems (procurement, finance, fulfillment), and scaling technologies (sysadmin activities).

In contrast, technical debt refers to systems and code that place strain on your product (in the form of bugs and scaling issues) and gets worse the longer it isn’t addressed. According to Steve McConnell there are two kinds of technical debt:

• Unintentional debt occurs when the wrong technical design is implemented, or a programmer just writes bad code. This kind of debt is non-strategic, and you want as little of it as possible.
• Intentional debt occurs when the organization knows that what they’re doing isn’t ideal, but it’s a compromise worth making for whatever reason (usually to do with budget or time constraints). Although not ideal, this kind of debt is inevitable in any organization. It needs to be minimized and dealt with, but unlike unintentional debt, it is not entirely unavoidable.

There are many obvious reasons to avoid, minimize, and pay down technical debt. But the most salient argument is to avoid what’s commonly referred to as the broken windows theory. This criminological theory aims to explain the effect of urban disorder and vandalism and states that:

“Maintaining and monitoring urban environments in a well-ordered condition may stop further vandalism and escalation into more serious crime.”

The basic analogy is that software is like an urban environment. As soon as a few broken windows (bad code) appear in the environment, and those windows are not immediately repaired, the tendency is for vandals to break a few more windows (we stop caring about good code). Then the environment starts to deteriorate: litter appears, squatters show up, and so on (all coding standards go out the window). Before long, all hell breaks loose. To quote Steve McConnell again:

If the debt grows large enough, eventually the company will spend more on servicing its debt than it invests in increasing the value of its other assets. A common example is a legacy code base in which so much work goes into keeping a production system running (i.e., ‘servicing the debt’) that there is little time left over to add new capabilities to the system.

That’s a situation that that needs to be avoided at all costs. Finding space on the roadmap to address technical debt is usually an extremely tough sell. Paying down debt is not sexy — you usually can’t see any changes in the front end, very few people understand what’s going on so there is a fair bit of skepticism about why the work is needed, and no one really wants to go clean up litter in the code. But it’s essential to prioritize technical debt in most, if not every, development cycle, to avoid the breakdown of basic services until there is nothing left but a ghost town.

It’s important to note that technical debt isn’t necessarily bad at the moment it occurs. Sometimes technical debt allows a feature-rich release to happen when it otherwise wouldn’t have if there was a zero-tolerance approach to technical debt. In general, new debt is OK, old debt is bad. Henrik Kniberg proposes a great way to deal with out of control technical debt in his article “Good and Bad Technical Debt”. He introduces the concept of a debt ceiling, where you stop to make sure the debt doesn’t get out of control:

When debt hits the ceiling, we declare “debt alert!”, the doors are closed, all new development stops, and everybody focuses on cleaning up the code until they’re all the way back down to the baseline.

Ideally you’ll address technical debt during every development cycle, but sometimes you’re going to hit that ceiling, and then it’s important to stop and work on it before it’s too late.

Putting It All Together

Gathering user needs, business needs, and technical needs is one thing. Figuring out what to do with all this information is something else entirely. This isn’t a post about prioritization, but for now, suffice to say that it’s important to strike a balance between addressing user, business, and technical needs, and that how this balance changes is based on three main factors:

• The stage of the product in its life cycle. Is it a brand new product, or has it been around for a while?
• The level of user engagement. Are you struggling to gain traction, or are users beating down the door to use your product?
• The financial state of the business. Are you still figuring out how to make money, or is there a steady revenue stream?

Depending on how those three factors are put together, you will have a different focus on your product development. Is the product brand new, and in a heavy acquisition phase? Then user needs should carry more weight. Is the business seeing massive organic growth? Then put more focus on scaling and revenue growth.

This approach gives us a rough idea of how priorities and needs should be balanced, but it’s a long way from figuring out what to build, and how and when to do it.

The point that deserves to be stressed is this: without doing the work to understand the core user, business, and technical needs that your product will address, you’ll be building a foundation on sand.

The product might work for a while, but eventually something better will come along. So instead of relying on dangerous assumptions, build a sustainable product on the solid rock of real insights.

Of course, uncovering needs is one thing. Figuring out how to turn those insights into a successful product is something else entirely. And that’s what the rest of the book is about…

Excerpted and abridged from Making It Right: Product Management For A Startup World by Rian Van Der Merwe.

Available for purchase from Amazon, iBooks, and Smashing Magazine.