In Coding, Fast and Slow, I talked about one of the deepest challenges in writing software: the near-total inability of developers to predict how long a project will take.

Fortunately, as that article mentioned, there is a way to work where the software you write ends up being valuable, and the business people you work with end up being happy. And critically, this way of working does not involve committing to estimates of how long work will take (which is good because I suck beyond all belief at such estimates… even for work that I initially believe will take no longer than a single day).

In a lot of ways, this is the most important thing I’ve learned in my (let’s say many) years of being paid to write software for people.

The core idea is: Put uncertainty and risk at the center of a conversation between the developers and the rest of the business (instead of everyone pretending such nasty things don’t exist). Doing so allows the entire business to tackle those genuine challenges together.

To show what such a conversation might look like, I will develop this approach in detail in the context of a story.

Welcome To <Company X>, Here’s Your Spec

Let’s say you’ve started leading a small team of engineers at a new job. On your first day, an Important Person comes by your desk. After some welcome-to-the-business chit-chat, they hand you a spec. You look it over — it describes a new report to add to the company’s product. Of course, like all specs, it’s pretty vague, and worse, it uses some jargon you’ve heard around the office but haven’t quite figured out yet.

You look up from the spec to discover that the Important Person is staring at you expectantly: “So, <Your Name>, do you think you and your team can get that done in three months?”

What do you do?

Here are some possible approaches (all of which I’ve tried… and none of which has ever worked out well):

• Immediately try to flesh out the spec in more detail

“How are we summing up this number? Is this piece of data required? What does <jargon word> mean, here, exactly?”

• Stall, and take the spec to your new team

“Hmm. Hmm. Hmmmmmmmm. Do you think, um, Bob (that’s his name, right?) has the best handle on these kinds of things?”

• Give the spec a quick skim, and then listen to the seductive voice of System I

“Sure, yeah, three months sounds reasonable” (OMG, I wish this wasn’t something I’ve done SO MANY TIMES).

• Push back aggressively

“I read this incredibly convincing blog post about how it’s impossible to commit to deadlines for software projects. Sorry, I just can’t do that.”

Here’s the thing about all of the above: they’re basically guaranteed to fail. By this, I mean specifically. No one is going to be any kind of happy about the software that gets written from the above starting points.

“Okay, Enough Stalling, So What Do I Do, Dan?”

I’m going to suggest something that may sound a bit odd — while this Important Person is standing at your desk, use this opportunity to, politely but ruthlessly, interrogate them about the business you have just joined.

• What is the business model?
• What are the biggest challenges facing the business as a whole?
• What risks does leadership worry most about?
• What are they hoping happens if everything goes just right?
• Who is the current customer for the product?
• What motivates that customer to buy?
• Are they happy after they buy? If not, why not?
• What other customers would the Important Person like to go after if they could?

One way to understand this is: there is some central problem or challenge which the business is facing. Your first job is to figure out what that problem is and, just as importantly, what words the Important Person uses when they think about it.

A very important thing: it usually takes a considerable bit of effort to get beyond the proposed solution (for example, the report) to the actual underlying problem. Laura Klein summarizes this marvelously, “[People] will tell you that they want a toaster in their car, when what they really mean is that they don’t have time to make breakfast in the morning.” She’s talking about user research, but I find the same perspective is incredibly useful when talking to, for example, CEOs.

Returning to our example, let’s say that, as you talk to the Important Person, you understand that your new business, which sells software via a monthly subscription plan, has a serious problem — too many customers are canceling every month. What’s more, you’ve joined a startup, and although it has a solid chunk of cash in the bank, the leaders very much want to ramp up how much they spend on sales and marketing. Of course, doing that will burn through their cash, thus requiring raising more capital sooner than later. And getting VCs to invest more money with that high cancellation rate will be very difficult, if not impossible.

You’ve been hired, at some level, to help solve that problem — even if the people who have hired you don’t think about it that way.

Now that you understand the central problem, take one more step: figure out exactly how this proposed development effort will solve that problem.

How and why does the business believe this report will lower the cancellation rate? What makes the Important Person think it’s going to work? Are there any ways they’re worried that it might not work? Are there any key questions they’d like answered sooner than later?

Oh, How People Love To Hear Their Own Words

A key tip for these conversations: at each step, it’s really helpful to echo back what the person just said to you. For example, “Okay, let me make sure I understand. You’re saying this new feature you want is critical because it’s going to help us upsell existing customers, but we’re not so much expecting it to help us get new customers? Do I have that right?”

At each of those little checkpoints, if you’re right, the Important Person will feel this rare, pleasant sense that someone in development actually seems to understand how the goddamn business works. If you’re wrong, you’ve just narrowly avoided basing your dev efforts on an imperfect understanding of the business (which is a path straight to misery).

Note that template: a) “I’m going to echo that back, make sure I understand,” b) echo it back, c) “Do I have that right?”. I say exactly those words nearly every time I talk to someone about a new project — so much so that my partner Edmund calls it “pulling a Milstein.” You don’t have to be clever with that template, is what I’m saying. Put all your cleverness to work really listening and trying to understand the problems facing the business.

This whole process takes practice but is INSANELY VALUABLE. You can (and should!) start by asking everyone you work with about how they understand the overall business you’re currently in and what challenges it’s facing. Do the same with random people you meet. Be curious, don’t stop being curious, and don’t be afraid to say, “I don’t understand that. Can you explain it to me?”

Now, The Knockout Punch

Once you both understand some central problem facing the overall business and how your proposed bit of development effort fits into a possible solution, you wrap all that up and deliver it back, repeating as many of the words they used as possible. For example:

“Okay, if I understand it properly, we’re adding this report because we think we can use it as a key feature in a new, higher pricing tier. This more expensive tier is not really for acquiring new customers; it’s more for upselling existing ones to extract more revenue from our most engaged customers. If we can do that, it’ll have a potentially big impact on our revenue churn, which is the most important number in our business right now. And we really need to see that move in the right direction in the next 6–9 months, so we’ve got a good story to tell investors when we go out to raise our next round of financing.

Do I have that mostly right?”

With even a modest bit of luck, at this point, the person who handed you the spec will have a cautiously hopeful expression on their face, and they’ll nod as they say, “Yeah, that’s… um… that’s pretty much exactly right.”

You then say, “Great, let me look into the tech we need for that report, and I’ll get back to you with more info.”

Note: You haven’t promised any date by which the report will be finished. Instead, you’ve demonstrated that you will work with this Important Person to solve the actual problems the business is facing. And those problems involve very real, very hard external deadlines (for example, running out of money by a certain date).

Here’s one way to see it: you’ve taken a key first step in earning their trust.

Now, notice too, instead of you making some promises to deliver on a spec — and those promises are now hanging over you and making you nervous — you’ve directly engaged in a real problem for the business. And you have plenty of room to be creative about solving that problem. Yes, it’s a hard problem, but that’s why you got into this business in the first place — for the joy of solving hard problems that matter to someone.

Man, If Only We Knew What To Do

The next day, you meet with the team and discover that the new report is mostly straightforward, except for one thing: it requires a periodic import of data from a new social network with a complex API. The team has just started working with that API, and they tell you they don’t have enough information to make the call on the three-month deadline. It’s certainly possible they could hit it, but there’s every chance things could blow up.

What do you do?

You could tell the Important Person that you don’t know. That is, at least, honest. But it doesn’t help them (aka help the business solve its problem — move revenue churn in the right direction before the next round of funding).

What would help you solve the business’s problem?

One key is that the business is trying to decide how to spend your time.

If you knew for certain that you could get the report built in three months (and that existing customers would happily pay more for it), the right decision for the business would be — build it.

Conversely, if you knew for certain that you couldn’t hit the deadline (or that existing customers wouldn’t pay more), the right decision would be — stop immediately, and start some other plan to reduce revenue churn.

Given that you don’t know which of those two alternatives you’re living in, you (and the business) need more information.

If you obtain that information, you could make the right decision, which would make your business much more money than the wrong one.

In the presence of uncertainty, acquiring information is often the best way to generate value. And, yes, this is the point in this article where I tell you to read Donald Reinertsen’s Principles of Product Development Flow.

So, what you do is pick what you work on next to gather as much information as possible about the things you are most uncertain about. If you’re clever (and you are! That’s why you got into development in the first place), you can find a way to gather information as part of building the thing. Meaning, you usually don’t need to conduct some separate, research-y phase. Instead, you can gather the information you need by doing your work in a careful sequence.

And, crucially, you must be completely upfront about all this with your counterpart on the business side.

The Meeting Where You Earn Your Salary

In our story, you schedule a meeting with the Important Person, and in advance of that meeting, you bury yourself in the technical details of what the team has found about that new API. You also do some chatting with the sales and marketing folks. You want to understand the target customer and which of their problems the business is hoping to solve with this new report.

Then, at that meeting with the Important Person, you say something like:

“Right now, we’re feeling optimistic that we’ll have that report ready in some form within three months, but our biggest risk is working with that new social network’s API. From the initial investigation, it looks like, at the very least, we’d definitely be able to show them <minimal data foo>. From what I understand of our engaged customers, this might be enough to trigger upsells — but sales and marketing aren’t certain.

We’d like to propose the following: Take two of our best devs and spend two weeks trying to build a full integration with the social network purely on its own. After this, we’ll have a better understanding of just how much data we can pull in. While they’re doing that, we’d also like to have our frontend devs build mockups of a report with just the minimal data so that you’ll have something to do user research on. You may be able to use the data for sales demos if things go well.

Does that plan sound like a good way to go?”

This little speech is the most important thing you’ll do at your job all month, so I want to unpack it in some detail.

First off, note that because you’re thinking in terms of risks and information, you propose sequencing the work to get as much information as quickly as possible.

For example, information about how much data you can get from the social network and whether customers will be satisfied by the minimal data set. When facing a chain of risks, you will first generate the most valuable information by attacking the biggest risks.

Second, it should be clear you can only pull this off if you deeply understand the overall business problem. That’s what lets you propose the minimal data thing. Generally, those opportunities emerge bottom up, as, for example, a dev figures out what data is or is not easy to obtain, but the value is not always clear to those devs. (The best way to run this game plan is to make it so that all the devs deeply understand the overall business problem.)

Third, it’s important that you’re offering the Important Person an actual, meaningful choice. You’ve clearly stated your current knowledge of what is possible (for example, the technical risks and opportunities), plus your current understanding of what is valuable (to the business). You’ve framed that in a way that lets the Important Person now choose what to do next (which will often result in you learning that your understanding of what is valuable to the rest of the business is no longer accurate — a very, very good thing).

Fourth, note that when you work this way, there are good risks (we call them “opportunities”) and bad ones. You discovered something unexpected — you could quickly and cheaply build a simpler report that might work. One of the most fun things about this approach is finding those wins; it’s tremendously exciting.

Finally, notice how you’re explicitly operating with full knowledge of the business’s hard, external deadline. You’re not talking about deadlines for implementing a spec but about deadlines for the overall business. These are the only ones that actually matter.

What Happens Next

Now, the Important Person might say any of the following:

a) “Great, go for it.” (You say, “Thanks, sir/madam, we’ll see you in two weeks with more information + options.”)

b) “That minimal data would be a fantastic report; I’m certain we can get upsells with that.” (You say, “Awesome, we won’t put the devs on an exploratory full backend integration. We’ll sprint ahead on getting the minimal data ready asap; we should have an early, crude prototype to look at it within a week or two.”)

c) “That minimal data is not enough.” (In which case, you say, “Okay, would you like to see other options for restricted data?” or “Hmm, I’d love to better understand what questions we’re trying to answer with this report since I don’t feel like I quite get it yet,” or even “Well, in that case, maybe we should explore some other options for reducing revenue churn in parallel because there’s a real chance we won’t be able to make this report work in time.”)

Note that that last situation is not, in any way, a failure. You’ve learned something very important — the business folks believe that the current plan centrally depends on something with a great deal of risk. Armed with this information, you can both try to drive down that risk as aggressively as possible and start working with them to prepare other plans, so you’re ready if things blow up.

Overall, this approach means you will constantly be adjusting your understanding of the most valuable way to spend your time and constantly keeping the business folks in the loop + offering them meaningful choices. This is not, in any way, “we don’t need no stinking estimates; we’re code cowboys. Just trust in the full force of our awesomeness.” It’s turning the entire software dev process into an ongoing conversation with the rest of the business — where information is quickly getting into the hands of people who can make decisions about it. And, where “information” means things you know/have learned and an understanding of what you don’t yet know — i.e., important risks.

As I said in my previous article, writing software means learning something in such precise detail that you can tell a computer how to do it. More broadly, if creating new software is important to a business, the business must engage in a learning process — not just the developers.

Hmmm, This Doesn’t Really Feel Like a “Process”

Inevitably, my solutions to this feel somewhat personal — but that is not an accident. Fundamentally, we’re talking about two groups of people having to build trust in each other. Trust about things they will not, generally, be able to verify.

Specifically, developers have to trust that what they are being told about the rest of the business is true — that customers want what they’re building, that the long hours are actually needed (and aren’t just some middle manager showing they know how to crack the whip. I wish I didn’t see that happen, but it does, far, far too often).

And the rest of the business has to trust that the developers, when they go off into their weird, opaque world, are honestly reporting back on what is possible, how much effort is involved, what they’ve achieved, etc.

Any means of building trust will always have a personal flavor — it exists between human beings who have learned something from each other. You cannot mandate or fix it with an imposed process.

Anyone who has done any real work on either side of that divide can immediately call up instances of betrayal — of discovering that all your work for the last half year was meaningless (and that someone knew that and didn’t tell you), or the repeated promises that some system was ready to launch collapsed in a fiery wreck as soon as the first user tried to log in.

Sometimes, the World Is Telling You To Polish Up Your LinkedIn Profile

A severe warning: This whole plan can fail badly if the Important Person is, well, not very important. Specifically, say you have a strongly hierarchical structure where some middle manager is the only person you can talk to. It can be the case that such a person perceives their job as taking proposed solutions from upper management and getting a bunch of developers to implement them. Such a person can be very threatened by the idea that you want to get beyond the proposed solution to the underlying problem. They can hear that as, “I’m going to have to go back to my boss and tell them that a bunch of developers think their idea isn’t very good.”

When a boss says, “Jump!” this kind of person prides themselves on saying, “How high?!” Since I’m instead proposing, “Why are we even jumping here?” you can see how there can be a problem.

Furthermore, such a person will often put a really strong value on preventing the flow of information (from developers to people who can actually make decisions). They may think of that as “Not troubling the boss with the details,” but as I’ve described above, such a block on the flow of information is deadly to software development.

So, what’s your best option if you find yourself in such an unfortunate situation?

As I see it, there are two paths ahead.

Option 1: Try to get the middle manager to see this new way of working as something that will make them look good.

I rarely see this work, but it can be worth a shot. My partner Edmund reports some success trying this by doing the following: a) finding an ‘internal’ thing, where the middle manager is, like, a user of the thing, b) proposing to them that you work on that internal thing this new way, and then, c) if that produces a thing they find really useful, help them see their boss can feel the way they feel now.

But, as mentioned above, that’s a long shot, which leads us to…

Option 2: Quit.

I don’t say this casually. If you’re stuck in the situation I’m describing, it’s overwhelmingly likely that your project will end in some form of unpleasant failure. And, what’s more, you can rarely get higher-level leadership to see any problems with such a middle manager. In general, such a person has that job precisely because they fit into higher-level leadership’s mental model of a manager. In which case, the entire organization is going to be set up in a way that makes it hard or even impossible to write useful/valuable software.

If you’ve been stuck in such a situation for a while, I’ll just say — you may have forgotten how great it feels to solve meaningful problems for people. Go find a place where you can do that.

Your Mission, Should You Choose To Accept It

In summary, I’m saying the following:

1. Become a student of the overall business you are in
2. Sequence your work to extract as much information from reality as early as possible
3. Make risks and opportunities the centerpiece of an ongoing conversation with the rest of the business.

There are no certainties in this world, but that approach will let you tackle the uncertainties together.

And that, I can tell you from fortunate experience, is a profoundly satisfying way to work.

But, Wait I Want To Learn More

I’ve stolen… ahem, synthesized just about all of the above ideas from a bunch of brilliant people. You should totally go read their books and blog posts. Here’s a few you might like:

Donald Reinertsen, Principles of Product Development Flow. If you a) love math and b) have spent ten years trying to figure out why your software projects keep getting cancelled, drop absolutely everything you’re doing and read Reinertsen right now. Otherwise, first read The Goal by Eliyahu M. Goldratt, and then read Reinertsen.

Eric Ries, The Lean Startup. He works out a very powerful set of ideas for generating value in conditions of extreme uncertainty. As you can tell from the name of his book, his focus is on startups, but I find his ideas broadly useful for software development in general.

Kent Beck, Software Design Glossary and Extreme Programming Explained. Few people have written as thoughtfully and intelligently about software development as Mr. Kent Beck. His work at the intersection of complexity, human nature, and economic value has greatly influenced me.

Douglas W. Hubbard, How to Measure Anything. His book provides some fascinating ideas on turning a vague statement like “We could make a better decision if we had more information” into something with concrete dollars attached to it. If you love math… you’ll wish he had written a shorter book with a lot more math, but such is life.

Laura Klein, Users Know and UX for Lean Startups. Truly great stuff on how to talk to human beings.

“So, wait, your article got so long that you included an appendix disguised as a series of links to more information?” In my defense, I can only quote a beloved one-time coworker: “No, so is your face.”