I came across Nolan Lawson's benchmark on the AI cost of JavaScript frameworks, and it's one of the more interesting framework comparisons I've read recently.
Instead of asking which framework developers prefer, Nolan asked an AI agent to build the same Pokémon application in 12 different stacks, then measured the time, tokens, cost, code size and Lighthouse results.
A few numbers caught my attention:
- Nuxt used the fewest tokens of any framework tested (3.85 million), including vanilla JavaScript. It also generated the second-smallest codebase, at 1,415 lines.
- Astro was the fastest to build (8.2 minutes), and the resulting application shipped a median of 0 KiB of client-side JavaScript.
- Angular took 17.2 minutes and 9.76 million tokens, generated 2,009 lines, and failed to produce a working app in two of five attempts.
These aren't universal rankings. It's a small, fairly content-heavy app, and the results vary between runs. A complex interactive dashboard could tell a very different story.
But having worked with both Nuxt and Astro, I found the results interesting. Astro's HTML-first, islands approach naturally limits unnecessary browser JavaScript. Nuxt provides different rendering modes for different kinds of pages. And Angular has incremental hydration, so its performance story is considerably more nuanced than a single benchmark suggests.
What surprised me most was the agent side of the results. Nuxt's MCP wasn't used in any of the runs, and Astro's was used in only two. Yet both did very well.
It makes me wonder whether we've been thinking about AI tooling backwards. Perhaps the biggest advantage isn't giving an agent more documentation, tools or instructions. It's having a framework with conventions that let it do the right thing without needing to ask.
We've spent years discussing developer experience. As agents write more of our code, there's another dimension worth measuring: agent experience.
Not just how quickly an agent can generate an application, but how much code it creates, how often it gets stuck, and what that means for every future change.
I'm not about to pick a framework based on one Pokémon app. But I do think Nolan is asking exactly the right question: if the way we write software is changing, shouldn't the way we evaluate our tools change too?