Speaking on stage at AWS re:Invent 2023, the Amazon CTO laid out the fundamentals for building cost-conscious software.

This year has put cost consciousness towards the top of engineering managers’ agenda. As interest rates soared, companies have tightened their belts, marking the end of a sustained period of hypergrowth for the technology sector.

Now, engineering teams are being tasked with digging into their codebase and looking for opportunities to reduce costs, especially as they relate to the cloud infrastructure many modern applications run on.

This was a theme picked up by Amazon’s CTO Werner Vogels last week during his AWS re:Invent keynote, a speaking slot normally characterized by unfettered bullishness about the growth of cloud computing and many, many product announcements. Instead, this year, Vogels was all about frugality, getting back to the basics of good cloud architecture and running applications on the most appropriate infrastructure.

“As always, when constraints get removed, when we throw off the shackles of something that keeps us down, we have a tendency to swing this pendulum all the way to the other side,” he said. “Suddenly the most important thing is to move fast, get products out, and do all the things you couldn’t before…but as speed of execution becomes more important we kind of lost this art of architecting for cost.”

During that keynote, Vogels laid out the following seven laws of cost-effective cloud architectures.

Design

The first three laws fall under the category of design.

1. Cost is a non-functional requirement: Vogels started by identifying cost and its close counterpart, sustainability, as non-functional requirements, alongside other core design considerations like security, compliance, accessibility, performance, availability, scalability, and maintainability.
2. Systems that last align cost to business: It may seem simple, but your costs should align most closely with where revenue is being driven. “Find the dimension you are going to make money over, then make sure that architecture follows the money,” Vogels wrote in a blog post on cost-aware architectures back in 2012.

   This requires the engineering function to align more closely than ever with the rest of the business. “We aren’t building technology for technology’s sake, we are building technology to support our business,” he said. “We need to have this conversation continually with our business partners.” Words worth remembering before getting carried away with the hype around AI.

3. Architecting is always a set of tradeoffs: Engineering involves tradeoffs, and cost is another consideration when designing a system. Cat Swetel, a senior engineering manager at Nubank, joined Vogels to speak about how the Brazilian neobank had found itself in a cycle of “throwing more machines, more memory at the problem,” instead of solving the problem, which was making sure systems were running on the relevant infrastructure. “Our hypothesis was if we stabilized our systems, costs would also stabilize,” she said.

Measure

The next two laws fall under the category of measurement.

1. Unobserved systems lead to unknown costs: You can’t improve what you can’t measure. With costs, this means understanding the actual cost of all of your software systems and where there are opportunities to improve them over time. “You need to understand what you are measuring and how that changes behavior,” Vogels said.
2. Cost-aware architectures implement cost controls: “You need to implement cost controls; you can’t just rely on good intentions,” Vogels said. “You need to build and have at your fingertips tuneable architectures.”

   Vogels explained that while engineers should “build the switch” to allow for certain components to be tuned according to cost considerations, “the switch should be in the hands of the business,” so that others can take direct responsibility for these decisions. This involves deciding as a business which tier certain services reside on and where you are willing to make tradeoffs that may impact customers.

   Examples of cost control methods include:

• Switching off features
• Throttling
• Turning off pre-fetching
• Reducing logs/metrics
• Showing fewer details

Optimize

The last two laws focus on optimization.

1. Cost optimization is incremental: Vogels described it as a “lost art,” where good cost-aware engineering organizations constantly seek to cut digital waste in their environments. Turning things off, right-sizing instances, or even reducing the capabilities you provide to customers if they aren’t vital are ways to start reducing wasted resources.
2. Unchallenged success leads to assumptions: “Please hold on to your egos,” Vogels said, “the most dangerous phrase in the English language is ‘we have always done it this way.’”

   The cost to build a service is nothing compared to the cost of operating it, so the “way you build” should be “continually under scrutiny,” Vogels said. By questioning deeply held engineering beliefs, you can start to uncover new cost-saving opportunities and keep moving your organization forwards.

Final thoughts

Speaking after the keynote, Mark Robinson, infrastructure engineer at Plaid and cloud cost expert, said he liked the focus on designing for efficiency but that “most engineers lack signal around it.”

The problem is that setting up “real-time cost and efficiency data is hard to get…so few places are set up to extract that kind of data or to even compare that long term,” he said.

While Vogels’ seven laws may look simple, and many of them are getting back to the basics of good engineering practice, it’s also vital that good cost observability is built in from the ground up so that engineers can make these decisions and trade-offs later down the line.