Introduction

Dig into Nick Caldwell’s resume, and you’ll see leadership posts at an enviable list of startup staples — Reddit, Looker, and now Twitter. But less typical these days is the 15-year stint he put in at Microsoft beforehand. His tenure at the company culminated as a founding member and eventual GM for Power BI, one of the company’s biggest success stories. It was only after accumulating this deep bedrock of experience that he took on his first startup role at Reddit, where he led the engineering team through hypergrowth, scaling from 35 engineers to 150.

He then joined Looker as a product and engineering exec, steering the teams through the company’s $2.6 billion acquisition by Google. This brings us to his latest role — as the VP of Engineering for Twitter, spearheading an org that’s 700 engineers strong.

It’s a remarkable rise with stops at some of the most interesting and innovative companies around. But what perhaps most sticks out about his career journey is that Caldwell has found success at companies with vastly different cultures, from Microsoft’s buttoned-up enterprise to Reddit’s fast pace. “Each place I’ve worked has radically different cultures, which I didn’t appreciate at the time because it’s difficult to understand the culture while you’re in it,” he says.

Part of the joy I’ve had over the course of my career is learning that there are a lot of different ways to ship software and get things done.

In our exclusive interview with Caldwell, he pulls on threads from each stop in his career journey at companies with different cultures, scales and functions and he opens up about the biggest leadership lessons that stick with him. He makes the case that Microsoft’s operational practices should get more of Silicon Valley’s spotlight — including the company’s approach to org design and its improved performance management system. He gets vulnerable about one of the harshest (yet warranted) pieces of feedback he ever received, and how a home run at Microsoft prompted him to leave the company and try his hand at startups. Caldwell also dives into his functional expertise, with his vision for how product and engineering can team up instead of tussle and the system he leaned on for architecting Looker’s engineering roadmap.

It’s an incredibly wide-reaching set of frameworks and although Caldwell’s bread and butter is engineering leadership, there’s plenty to sink your teeth into for managers all over the org chart. Caldwell is a practiced speaker and writer, and he’s got a knack for distilling complicated topics or thorny impasses into crystal-clear lessons. Let’s dive in.

LESSON #1: TAKE RESPONSIBILITY FOR WHAT HAPPENS NEXT & GET OFF THE FLOOR.

Caldwell points to two of his former managers at Microsoft for some of his earliest and most pivotal leadership lessons. “When I was very early in my career, I was a brash, headstrong engineer. I was complaining to my manager at the time, Ravi Shahani, about how crappy the PM team was,” recalls Caldwell. “I was like, ‘These guys don’t know what they’re doing, they’re making me build all this stuff and none of it’s going to work.’ I was just going off on him.”

What came next was a (somewhat painful) wake-up call. “He said, ‘Nick, you’re an amazing engineer, but you’re just a shitty leader.’ I was like, ‘Ouch!’” says Caldwell. But Shahani imparted a critical piece of advice. “He said, ‘Leaders take responsibility for what happens next. So are you going to sit in my office complaining about this, or are you going to do something to change it?’ And that was an incredibly important turning point in my career — it was the moment I decided I was going to do more than just be a grunt engineer,” he says.

His next move? “I immediately left his office and went to the product’s GM to say, ‘Hey, we need to change this in the product strategy — give me a chance to prove it to you,” he says.

Another leadership lesson came a few years later when Caldwell had climbed from IC engineer to engineering manager, and eventually engineering director. “James Phillips was my GM at the time, and he taught me a lot about the difference between being a manager and a director,” says Caldwell.

“Up until this point in my career, I loved line-level engineering management. I carried that through to my role as a director, even though I had like 30 people reporting to me. I did all the things that you would from someone with a much smaller team — I got to know everyone individually, and worked hard to motivate and inspire people on a one-to-one basis. I was having trouble scaling all of the things that are great about being an EM,” says Caldwell.

To close the gap, Phillips gave him a mantra that Caldwell now passes along to other folks making the manager-to-director transition. “One day James said, ‘Nick, you have to learn how to get off the floor.’ What he meant by that was if you think about the engineering team as a warehouse production, you’ve got your line managers and then you’ve got your directors. The directors have to be off the floor, overseeing multiple lines,” he says.

Phillips took an additional step to hammer home his point. “He made me stop sitting with the teams directly to force me to get a bit of removal from the day-to-day. It did teach me how to delegate more effectively,” says Caldwell.

When you get off the floor, you can see systematically how your teams are working together and better spot the bottlenecks from that vantage point.

LESSON #2: ACTUALLY, YOU SHOULD SHIP YOUR ORG CHART.

While the cultural practices of FAANG companies have permeated nearly every company in Silicon Valley — from early-stage startups to industry juggernauts — Microsoft’s operating principles haven’t captured the zeitgeist. That’s a missed opportunity, says Caldwell. “Google’s management practices and Netflix’s culture deck are held on a high pedestal. But my honest assessment is that they pale in comparison to the effectiveness of Microsoft’s organizational design,” he says. “Any other place I’ve worked relative to Microsoft has been less effective at forming business units and understanding how to move people and resources to whatever the most urgent problems are. It doesn’t get talked about much publicly, but it’s Microsoft’s cultural superpower.”

Here’s how he breaks down the company’s product-based org structure. “Windows, Cloud, Gaming or Office are all generally speaking multi-billion dollar businesses with even larger partner networks. So with that framing in mind, the company needs to organize its resources and people to address each disparate business’s needs. Each business unit is run by a VP, which then breaks down into product families, and within those product families are product units, and this trickles further down into technology teams and feature teams,” says Caldwell.

The word “reorg” may elicit groans from plenty of folks, but Caldwell points to this process as one of Microsoft’s most powerful tools for rigorous prioritization. “Microsoft on a very regular cadence changes the org chart, whether it’s where people sit, which products are owned by a particular PM, et cetera,” he says.

But it’s not just the org (and reorg) design that sets the tone here — an air-tight comms strategy is key. “You can expect that every quarter or every other quarter there will be some sort of reorg announced by your executive. It’s communicated very clearly and backed up by explaining what the business strategy is — whether that’s moving people to shore up investment in a particular area. There’s a regular cadence of pruning and shaping the org chart to match whatever the goals are that we’re trying to hit,” says Caldwell.

“Don’t ship your org chart” is a common saying. But this is incorrect — you should be shipping your org chart to make sure it maps to whatever product or objective the business is trying to achieve.

LESSON #3: CODE CAN’T SOLVE EVERYTHING.

While Caldwell’s primarily been an engineering leader, he’s also taken on product leadership — including merging the two as Looker’s Chief Product and Engineering Officer. It brings him a unique perspective on how these two orgs can avoid the often at-odds, “frenemies” relationship.

“I started in engineering and really had no clue what product people were supposed to be doing — I thought that code solved everything. Obviously, we can’t ship anything without code in our line of business, but I over-weighted it because I didn’t understand everything that went into getting the software out the door and into the customer’s hands,” says Caldwell.

Here’s how he now delineates between the two. “Product’s job is to consider a wide variety of inputs — customers, market strategy, going across organizational boundaries — and synthesize that into a winning strategy. Then they influence everyone from engineering, product marketing managers, go-to-market to agree with that strategy,” he says.

He’s got a greater appreciation for the many potholes along the way. “The challenge here is that each of these different groups has completely different motivations. PMMs think very differently than engineers, who think very differently than designers, and they’re incentivized in different ways,” says Caldwell. “They have different rubrics for how they judge their success, and product people have to sit in the middle of all that and figure out a way to corral it all towards whatever your product’s goal might be. It’s a maddening, multi-dimensional, never-ending chess game.”

On the engineering front, Caldwell finds that the biggest trap is leaving engineers out of the decision-making phase. “Engineers are often asked to go work on things without explaining why they matter to the business. Like, ‘Go build this feature.’ Or they’re asked to keep legacy code alive without understanding why or how it contributes,” he says.

Many companies think of engineers in a way that’s like, “Lock them in the basement, throw a pizza down there every once in a while, and hope that code comes out.” But every minute you spend explaining the “why” to an engineer is a minute well-spent.

Here’s his recipe for a happy marriage between product and eng orgs: “The best output is when engineers can trust that the PMs have a great understanding of the market and how the eng work ladders up to the customer. Meanwhile, the PMs look outward and try to collect and synthesize all the information. Their skillset has to be balancing data and intuition to come up with the right roadmap — and then explain that in a way that resonates across multiple functions,” says Caldwell.

It relies on a relentless focus on the customer. “When I joined Looker, it was a very engineering-focused product, and I remember there was a saying on the engineering team that said, ‘The engineer has the pen.’ That meant that the engineers on the team had the best understanding of what needed to be built,” he says. “I changed that attitude pretty quickly toward, ‘The customer has the pen.’”

LESSON #4: WEIGH THE PROS AND CONS OF INTRAPRENEURSHIP AND ENTREPRENEURSHIP.

Despite working for such a large enterprise company during his time at Microsoft, Caldwell got a taste of building from zero to one, although he admits it’s not quite the same as a true startup. “Any large company is going to have a challenge with zero to one projects in general. Because anything you build could impact existing business lines,” he says.

Here’s how it played out for Caldwell: “I was on the team that built Microsoft’s Power BI product. In the early days when we were talking about building a new data visualization tool, we had to go and clear that idea with multiple teams that would be impacted to even get moving on the concept,” he says. After clearing the initial concept, it wasn’t exactly off to the races. “It’s not a one-and-done moment to overcome inertia. You have to make time in your schedule and product roadmap to continuously clear ideas with other teams,” says Caldwell.

Despite the hurdles, Power BI eventually became a massive success for the company — which led to an impasse for Caldwell. “Microsoft does a phenomenal job of structuring a career — there are clear career ladders and they allow for a ton of employee mobility so you get exposure to a wide variety of different opportunities. I was working on natural language, machine learning, search, gaming — all sorts of different things. They make it a very easy place to spend your whole career there,” he says.

If you only work in one place for a long time, it becomes your world. The execs at the company become your heroes. You get wedded to the tech stack. Your understanding of the market is influenced by the products around you. You start to live in a bubble.

Caldwell realized he had been overweighting the wrong things. “I had been bouncing around to all of these different intrapreneurial projects within Microsoft. I was always focused on the downside of joining a startup. Like, ‘Hey, if this thing doesn’t work out, at least I’ll still be at Microsoft with my safe, cushy job.’ But I’d never deeply considered the upside of joining a startup if we build a hit multi-billion dollar business,” he says.

Here’s what caused the shift in his mindset: “Power BI ended up being colossally successful for Microsoft, but it wasn’t earth-shattering for me personally. If I had done this as a true entrepreneur, it would have been a phenomenal victory. I realized I was self-limiting by only living within that one ecosystem, which had been very comforting for so long. That’s what caused me to start souring on intrapreneurship and focus more on true entrepreneurship,” he says. It’s a turning point that led to his next move, taking the leap into the startup scene and joining Reddit as its VP of Engineering.

LESSON #5: USE PERFORMANCE REVIEWS TO CAPTURE LONG-TAIL EVOLUTION, NOT A SINGLE MOMENT IN TIME.

Reflecting back on his tenure at Microsoft, Caldwell saw plenty of other changes beyond consistent reorgs — including the dismantling of one infamous system. “Microsoft used to have a really atrocious stack-ranked performance review system, which they got rid of 7-8 years ago. They replaced it with what I’ve come to believe is the simplest, most straightforward, and equitable performance review system I’ve seen since. I’ve tried to reproduce it in every place that I’ve worked,” he says.

In the new-and-improved system, you begin by outlining OKRs and personal development goals for the year, alongside input from your manager. Every other month, you have a check-in with your manager on progress so far, plus a formal review at the six-month mark that’s not tied to compensation, with a final review at the end of the year that can result in additional bonus compensation for high performers.

Here’s why Caldwell believes it works so well: “Because those regular check-ins are happening every other month, you’ve got a trajectory that’s collected over time. It’s not just reviewing a long list of goals every 12 months or checking in on OKRs at one point in time — it’s really getting an assessment of your momentum with a lot of additional signals,” he says.

On the compensation front, managers have the discretion to spend additional budget on their top performers, with a deep level of oversight from HR. “When you make your budget recommendations as a manager, they roll up to the director and eventually the VP. HR overlays their checks on top of this, so promotions are being distributed evenly and diverse employees are not being overlooked,” says Caldwell. “Or if you’ve got a new manager with a different assessment of your performance, HR can weigh in with the historical data of your bi-monthly and bi-annual check-ins trajectory.”

His advice for startup leaders as they craft their own performance review processes? Consider how you can implement a regular drumbeat of feedback, rather than a one-and-done approach. And don’t forget to layer on checks and balances so no one’s overlooked.

LESSON #6: FOR A WINNING MINDSET, KEEP A CLOSE EYE ON COMPETITORS.

Caldwell points to another strength of Microsoft that he believes folks often overlook. “Microsoft was relentless in terms of understanding its competitors in the market. When I talk to early founders (or even some experienced founders), I think they overweight technical cleverness or a product idea without fully understanding what’s come before or how other competitors are situated in the market,” says Caldwell.

“Microsoft had entire teams of people doing nothing but understanding competition and crafting the right strategy to come out on top. That level of investment is not necessarily feasible at a startup, but the more you can understand the landscape, the more effective you’re going to be.”

I often hear advice from folks in the startup community that you have to power through and ignore your competition. But you need to understand how your competitors are positioning themselves so you can carve out your own niche.

It’s a mindset that’s stuck with Caldwell many years after he left Microsoft. “My way of running product and engineering comes from my time at Microsoft — I like to win. If you drop into any organization that I’m running, you’ll see that we spend every quarter doing an assessment of our strategy and the competition. What game are we playing, who are we playing against, and what does winning look like?” he says.

Here’s what that looks like in practice: “What are the tactical things that we are going to do to get closer to victory, and how can we measure success? Once we’ve agreed upon all of that, the next question is if we have the right organizational alignment, not just across engineering and product, but all functions of the company, to achieve our goal,” says Caldwell.

LESSON #7: STICK TO THREE-MONTH ROADMAPS.

When Caldwell joined Looker in 2018, the company had been humming along after emerging from stealth five years earlier (for a ton of details on what happened in between, you’ve got to dive into the Review article with co-founders Lloyd Tabb and Ben Porterfield). “I was very explicitly brought on to set up an R&D cadence because the team hadn’t really been operating with a clear roadmap. Very quickly I determined that we needed to add some understanding of what a roadmap is, what value it has, and build some systems for developing and maintaining that roadmap,” he says.

He outlines how he got started. “The framing I used was that we’re going to start from the top-down exec layer. I outline the strategic landscape as I see it, and describe in one page some of the opportunities that I think are ahead of us. For Looker, that might’ve been addressing some of the technical debt, moving toward a developer platform, a few other things — but keeping it at a super high-level framing,” says Caldwell.

Next, he brings in folks further down the org chart. “Then, over a two or three-week period, I asked the teams to come up with ideas and strategies that contribute towards that high-level set of goals (or push back against them). This would become a set of strategy memos which outlined how we’re going to win,” he says.

He then returned to his Microsoft roots, with a rigorous focus on aligning the org chart to match up with the strategy. “Do we have the right setup and the right resources? Do we have the key leaders in place to meet each of these objectives? It’s giving folks an opportunity to move to the areas that they are most passionate about. Or for projects that were a little more controversial, like changing out our visualization tech stack, do we need to look outside for folks that are passionate about that bet?” says Caldwell.

The strategy memos were then converted into roadmaps, filled in with specific items that will help achieve the overall goal — and Caldwell’s got a strict time frame for his roadmaps. “I like to run teams that have predictability. Anything further down than three months, it’s harder to accurately predict what’s achievable and I want to ensure we’ve got a clear picture of what we’ll deliver to the customer within a quarter,” he says.

Then the clock starts, with a weekly cadence for tracking progress towards those goals. “I sat with all of the drivers across each of these bets once a week, and we looked at progress towards every single item on the roadmap,” he says. His tip? You may eventually move to a bi-weekly check-in — but start with more frequent meetings as the team gets used to the roadmap framework. “At the end of the quarter, you retro, you optimize, and you do it again like clockwork,” says Caldwell.

LESSON #8: BRING OTHERS ALONG WITH YOU AS YOU CLIMB.

Caldwell was at Reddit for just about two years, but his relationship with the company’s co-founder Alexis Ohanian left an indelible mark. “When I joined Reddit and met Alexis for the first time at one of the board meetings, he told me, ‘Nick, I’m here for you. I’m going to support your career,” says Caldwell.

Looking back, he admits he wrote the interaction off at first. “We had literally just met, so I thought, ‘Yeah, everyone says that.’ But Alexis has been a phenomenally supportive person, not only for my career at Reddit but later on in encouraging me to become an investor. I’m now a fairly well-established angel investor. He helped with my referral to the Chief Product and Engineering Officer role at Looker. He helped with my recommendations for public board roles,” he says (Caldwell sits on the board for True Search and HubSpot). “He has, in every way, shape, or form kept an eye out for me in ways that cause me to often ask myself why.” (We got a chance to interview Ohanian for The Review back when he was still at the company, and it’s chock-full of wisdom well worth returning to.)

It’s a selfless spirit Caldwell tries to embody in his own tech career. “What I’ve taken away from Alexis is that you need to find ways to lift others as you climb. Whether it’s through my involvement in the nonprofit DevColor, or the educational blogs and videos that I do, to find ways to bring other folks along with me and look out for the next person,” he says.

This article is a lightly-edited summary of Nick Caldwell’s appearance on our new podcast, “In Depth.” If you haven’t listened to our show yet, be sure to check it out here.

Cover image by Getty Images / laremenko.