Engineering leadership · Book notes
Escaping the Build Trap when building costs almost nothing
Melissa Perri's build trap gets cheaper to fall into once AI makes the building part nearly free.
At Y Combinator’s AI Startup School talk last summer, Andrew Ng argued that AI coding tools make engineers go much faster, but that deciding what to build doesn’t go any faster, so now product management is the bottleneck. One of his teams proposed a ratio of 1 PM per half an engineer (instead of the normal ratio of 1 PM : 4 engineers).
He also said an experienced product manager’s gut is a surprisingly good substitute for data when you have to decide quickly.
This made me think of a book. I’ve been on both sides of the backlog: first as a software developer, and later as a software development manager. On a lot of teams, the scoreboard is the number of features shipped. Melissa Perri, in Escaping the Build Trap (O’Reilly, 2018), calls this the build trap: when an organization measures its success by output (features shipped) and loses sight of the outcomes (the customer value those features create).
This idea predates the book.. the original post from 2014 describes the build trap. Perri writes, “Building is the easy part of the product development process. Figuring out what to build and how we are going to build it is the hard part.” She suggests that we stop gauging productivity based on output and instead reward teams for learning more about their customers through research and experiments.
I think what Ng’s talk represents is Perri’s line finally coming true.. building is now actually easy. But I don’t think that means the build trap goes away. I think if more product managers make decisions based on their gut, that’s still a build trap if the team keeps measuring how many things they shipped. I think it makes it easier to get into the build trap, because if you have a coding assistant that will just build the next feature you ask it to every time, and never question whether that feature should exist in the first place, then even though the output is cheaper to produce, if you’re still measuring the team the same way, they’ll just make more unnecessary work, but faster.
But cheapness is also the escape. Perri described the escape years before AI made building cheap. She said treat every solution like an experiment in service of an outcome. Experiments are easier if building a working prototype costs a day instead of a quarter. So for every cheap build, the question is: how will we know it worked? What outcome should this move, and what will we do if it doesn’t? Otherwise, we’ll just be efficiently ineffective, shipping fast toward nothing in particular.
One question remains, which Perri’s framing from 2014 doesn’t need to answer. If building is near-free, and any idea can become a working feature by the end of the week, is the hard part still deciding what to build? Or is it deciding what to delete?