How to beat Amdahl's law with AI

Amdahl's law states that the overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used, and is often invoked by engineers in the context of AI productivity gains.

Translated into the AI productivity debate, the argument goes something like this:

Actual programming is only a small fraction of the software development process, so even if you have infinite speedup in the coding phase, you can only marginally improve the productivity of the whole process.

This is correct, but it's a pretty narrow view which assumes the best we can do with AI is to churn out code faster. We can do better, by thinking outside the box. To beat Amdahl's law, we can tweak constraints other than "optimizing a single part of a system".

Coding speedup

Let's first estimate the productivity increase for the single phase—coding—so we work with realistic numbers.

If we just compare human typing with token outputs from the LLM, the speedup is roughly 100x–1000x. However, this is obviously unrealistic. The programmer still needs to prompt the LLM, point it to relevant documentation, code, constraints, or other information. The AI system needs to call external tools to check or compile the code, run tests, and so on. There will inevitably be errors, retries, and multiple passes.

So something like 10x–20x seems more appropriate. Note that this is still for code changes that the developer never reviews. If they do even a cursory review of the implementation and spend time only on more important areas, that easily brings down the estimate to 5x.

These are just Fermi-style orders-of-magnitude estimates, but 5x coding speedup while staying mostly aware of what the code is doing and how it works is not unheard of.

Now we get to Amdahl: coding is only one of the phases in a software-development lifecycle (SDLC), next to planning, system and UX design, testing/QA, deployment, and maintenance.

If coding takes up only 25% of the time on a software task (a figure close to some estimates being thrown around), the 80% time saving on coding (equivalent to 5x speedup) only saves 20% for the entire task. That's a far cry from the "10x" improvement often touted by proponents of AI coding.

Here, I want to pause and say that 20% overall improvement is not a trivial achievement! For a team of 5, it amounts to one full-time extra developer. How good a deal that is also depends on the costs (tokens aren't free), but it's nothing to sneeze at.

However, we can do better.

Cheating Amdahl

Nothing stops us from beating Amdahl's law by moving work around to increase the percentage of work done by one component. Or in software development terms—let the programmer do more, not just coding.

What are some natural ways to expand the scope of software development roles that could take advantage of AI?

Here are a few ideas:

Requirements gathering and task analysis: these research tasks can take an inordinate amount of time on a mature software system. Investigating incoming feature requests, requirements, bugs to be fixed, poring over documentation for 3rd-party systems one needs to integrate, and looking into the current state of the code base could previously take days or weeks. Now, AI can help the product owner or developer do this faster and better.

Prototyping: instead of building the wrong thing or endlessly discussing minutiae of a task, AI can quickly whip up a working prototype. Doing a prototype first has been advocated since the dawn of computer engineering (Fred Brooks' plan to throw one away), but is skipped for expediency. This can lead to the task needing rework or rethink at the end of the entire lifecycle.

Automated review: use AI for a first-pass code review that catches the easy-to-spot bugs, to minimize the number of back-and-forth handoffs between the programmer and the code reviewer. The AI review might be enough for small, trivial changes, completely eliminating the costly and flow-destroying context switches. Larger changes should still involve a human reviewer (other than the programmer). All changes should still be reviewed by the programmer working with the AI on the task.

Invest productivity gains into higher quality: Some—maybe even most—productivity gains in the coding phase should be reinvested into making the codebase better: improve quality, pay off technical debt, fix long-standing paper cuts, maybe finally get to that low-priority refactor that never could get allocated the time before.

Improve tooling and processes: writing tools to achieve slight productivity gains across the whole process was historically a bad use of expensive developers' time. Not any more. These can often be vibe-coded quickly and with an acceptable level of quality for small quality-of-life and productivity improvements that add up over time.

Sensible software engineering

None of these are especially innovative or groundbreaking ideas. In fact, developers unsuccessfully argued for most of these long before AI came on the scene. You could just say it's sensible software engineering, not "cheating" Amdahl's premise.

You'd be right. There's no cheating the math formula. But when a math formula becomes a straitjacket preventing people from seeing the possibilities, it can be useful to step back and see if we can bend or break the assumptions behind it.