We watched developers approve bugs in 30 seconds

Recently, we turned code review into a speed competition at CodeRabbit’s app.js conference booth. At app.js, JS Nation, and React Summit, we asked hundreds of developers how they really review code. The answers were a lot less funny than the game.

The rules were simple. A code snippet hits the screen, and attendees have thirty seconds to approve or request changes from their phone.

Faster correct answers score more, so you're deciding under pressure. Once the votes lock, we reveal the bugs missed and their fixes.

Every round, we called out the current leader and the fastest correct answer, so way more people got a "THAT WAS ME" moment than just the three who'd actually take home prizes.

Round after round the same thing kept happening: a bug would go up, the clock would run, and a chunk of the room would confidently wave it through. The "obviousness" of a bug turned out to have almost nothing to do with whether people caught it at speed.

Then we talked to you

During those three conferences but also between game rounds we talked with hundreds of developers about how you actually review code. The same themes came up again and again and they're worth recapping, because most of them contradict how teams think review works.

It's built right in

The single most common setup we heard: whatever review bot came bundled with their code host, because it's already in the PR and it's one less tab to open.

We get the logic. It's the office-fridge sandwich of code review. It's right there, and the deli is a whole walk away. But these were often the same developers who told us, in the same conversation, that review is one of their bottlenecks to shipping.

If something is genuinely slowing your team down, "it came pre-installed" is a strange bar to set for the tool that's supposed to fix it.

We think that bar is worth raising, and that code reviews should read like a teammate left them instead of a linter with a thesaurus, about the parts that bent to fit their repo instead of the other way around. Everything for code review should be built around review quality.

The agent said it's fine, so it's fine

A good chunk of the developers we spoke with review code with the same coding agent that writes the code. Not as a dedicated review step; the tool is already open, it wrote the code, so it gets to check the code too.

What worries us is what happens after the agent answers: nothing. It hands down a verdict and the developer takes it. No pushback, no follow-up questions, no seam to pry open. The whole interaction runs in one direction.

So judgment gets outsourced to a tool that was built to generate code, not to have a conversation about a pull request, and then even that conversation stops.

Review is supposed to be a back-and-forth. Take it as-is and what you're holding is a fortune cookie.

I could build a better, cheaper version myself

Then there's the developer that's going to build it in-house. The model's right there, the API is cheap, how hard can it be? A day here and there and they'll have a reviewer tailored to their stack, at a fraction of what the dedicated tools charge.

It is not an easy task.

We're not going to relitigate the whole thing here, because we already did, in detail: your internal AI code review tool costs more than you think. The short version is that the model call is the cheap part. The expensive part is everything around it: the context engineering, the noise suppression so it doesn't flag every line of the diff, the integrations, the maintenance, the engineer-months now going into a code review tool instead of the product you're actually paid to ship. The "fraction of the cost" math only works if you never assign it a salary.

Building your own is the most expensive way to learn that the hard part was never the AI.

What all developers at the conferences agreed on

Strip away the tooling arguments, there was one belief that nobody disagreed with:

We ship too much code to review it all by hand. That ship has sailed, and we aren’t going back to that approach.

The volume of code generated by agents is up and to the right, and it isn’t declining anytime soon. Developers are now manually writing less code, but reviewing far more than ever before. That is the job now for developers, and it’s a hard job. At times it is a cognitive overload, as the diffs are getting larger, but we are still reviewing code in platforms that were meant for human generated code and not AI generated code.

AI code review isn't a nice-to-have your team adopts when it's mature enough. It's the load-bearing wall of how software gets shipped now. That's the category we set out to define, and it's the rare thing everyone at a conference already agrees with before you finish the sentence.