Every PR has a crowd of reviewers now and none of them know why
Automated review hits every pull request now, but none of it knows why the change exists or how much a problem matters.
In my career I have written code, reviewed code, and reviewed and approved and rejected PRs all by hand in the past. Now, when you make a PR on open source repositories you have all kinds of automated feedback hitting from CodeRabbit, GitHub Copilot, Claude, many others with system diagrams, analysis, breaking change analysis, security analysis all on top of the deterministic checks that we’ve all become accustomed to.
You can have many reviewers on a single pull request. CodeRabbit and GitHub Copilot code review can both comment before you even look at the diff as a human. Most of the feedback from the bots is good.
The issue with agents is that they do not know the weight of things. They can’t contextualize how severe an issue is.
This is where the human comes in. Severity is the human reviewer’s contribution.. the bot will flag a missing null check and a broken billing path in identical tone. A human still needs to decide which ones actually block the merge.
Thats just reviewing the code. I agree that this works well, but it also doesnt really scale, its a very long process. Everything that you will question can open a whole discussion, and before you realize you end up spending hours to review code that already works but is just not perfect.
Frankly, if your agent is doing 100k lines you are working on too large of an increments.
Before you even open the PR.. keep your changes small.
It doesn’t have to be like that if you had a ticketing system (GitHub issue?) that acts as spec or PRD
A review bot without a ticket can only review the code that’s given, it can’t tell if the change is the one that was requested. My agents operate under review gates for the same reason (see Running an agent fleet like a small team). They propose, I rule.
What I know for certain is that LLMs are infinite text spew machines. If you give them an excuse to spew more text they will.
Having an extra review bot on every PR just gives you more text to read and there’s no signal to tell you which comment is important. The job I used to do manually was read the lines. Now I get to provide the intent and decide what’s important. It still helps to write down your review rules, as I argued in Looks Good to Me and the case for writing down review rules.