How AI Code Review Actually Works

AI code review uses a language model to examine a code change, look for defects and explain its findings. Surrounding code and repository rules help the model distinguish a real bug from a convincing guess. The AI code review tools guide compares the products.

A missing tenant filter can let a user read another organisation's invoice. Following that bug from the changed code to the review finding shows where an AI reviewer can catch it and where the reviewer can fail.

When I judge a reviewer, I want to know whether it saw the relevant code and whether its finding reproduces, long before I ask which model it runs.

Performance figures published by vendors are not independent test results.

What Is AI Code Review?

An AI code reviewer is a service that reads a proposed change, gathers some surrounding context, asks one or more language models to find problems, and posts the results where the team already reviews code. Most install as an app on GitHub, and several also support GitLab, Bitbucket, or Azure DevOps. Some also run locally before a pull request exists.

A new engineer on your team needs the diff, a tour of the codebase and the team's review rules before their feedback is worth acting on. An AI reviewer needs the same three inputs, but it can still invent a convincing explanation after receiving all three.

AI review differs from both static analysis and human review. A linter or SAST scanner applies fixed rules and returns the same result for the same input and configuration. A human reviewer brings judgement and knows the product. The AI reviewer uses a model to reason about the change, but its answers can vary between runs.

A coding agent is the other neighbour. Codex and Claude Code write code and can also review it, while dedicated products such as CodeRabbit and Greptile are built around the pull request itself. The guide to coding agents as reviewers covers that split in depth.

Useful code review terms:

• Diff: the lines a pull request adds, removes, or changes, plus a few lines of surrounding context.
• Context: anything beyond the diff that the model gets to read, from neighbouring files to linked tickets.
• Rules file: a markdown or config file in the repository that tells the reviewer what your team cares about.
• Finding: one reported problem, usually tied to a line and tagged with a severity.
• False positive: a claimed defect that is not actually present, perhaps because a guard outside the reviewed context already prevents it.

How Does an AI Code Review Work?

A typical AI pull request review fetches the diff, gathers relevant context, applies review instructions and filters candidate findings before posting comments. Some products also run analysis tools or verify findings with additional agents. This example adds an invoice lookup beside an existing function that lists invoices for an organisation.

--- a/billing/invoices.py
+++ b/billing/invoices.py
@@ -21,3 +21,9 @@ def list_invoices(org_id):
         "SELECT id, total FROM invoices WHERE org_id = ?",
         (org_id,),
     ).fetchall()
+
+def get_invoice(invoice_id, org_id):
+    return db.execute(
+        "SELECT id, org_id, total FROM invoices WHERE id = ?",
+        (invoice_id,),
+    ).fetchone()

That snippet is the new hire's day one. The query is parameterised, so it's safe from SQL injection, and nothing in the added lines looks broken. The bug is that org_id arrives as an argument and never reaches the query, so any user can read any organisation's invoice by guessing an ID.

A diff-only reviewer sees the neighbouring WHERE org_id = ? filter just because Git includes three lines of context by default. Move the new function to another file and that clue disappears. The rest of the pipeline exists to put the clue back on purpose:

1. A trigger fires when the pull request opens, gets a push, or receives a mention such as @codex review.
2. The service fetches the diff and decides which changed files are in scope.
3. It gathers context from the repository, an index, linked tickets, or earlier review comments.
4. It loads your rules files and any learned preferences for this repository.
5. Some products run linters and security scanners and pass their output to the model.
6. One or more model passes produce candidate findings, and a filter or verification step drops weak ones.
7. The remaining findings become inline comments, a summary, and often a check run on the pull request.
8. Reactions and replies feed back into what the reviewer flags next time.

What Does an AI Reviewer Need to Check Your Code?

An AI reviewer needs the changed code, enough surrounding context to understand its effects, and clear instructions about repository-specific risks. The following stages explain how products gather and use those inputs, and where a missed finding can originate.

Triggers and the Diff

The trigger decides how often you pay and how much of the change gets read. GitHub Copilot, by default, reviews a pull request only when someone requests it, and users, repository owners or organisation owners can switch on automatic review. Unless it's configured to review each push, Copilot's automatic review runs once per pull request.

BugBot takes a different default for repeat reviews. Cursor's documentation says it reviews only the changes since its previous review, and you can switch off Incremental Review to reread the full diff on every push. That saves money, though a bug that spans an early commit and a late one may never sit inside a single review window.

Scope rules then trim the diff before any model reads it. Copilot's documentation lists file types it never reviews, including lockfiles such as package-lock.json and Cargo.lock, plus requirements.txt and tsconfig.json.

A pull request that only bumps a dependency gives Copilot almost nothing to read, which is one reason dependency risk belongs with tools like Dependabot and Socket.

Context Gathering and Indexing

Context is the codebase tour for the new hire, and vendors take very different routes to it.

Greptile builds a graph of the whole repository when you sign up, then queries it during each review for the callers and look-alike functions elsewhere in the code. Its own docs show it comparing a changed SQL helper with sibling functions to check whether each one uses parameterised queries.

CodeRabbit clones the full repository into a sandboxed cloud environment for each review and runs agents that explore it for context. Its FAQ also describes code indexing that stores vector representations of your code, using an approach called embeddings. Code chunks become numeric vectors, and the reviewer retrieves the chunks closest in meaning to the change.

GitHub Copilot gathers what GitHub calls full project context through agentic tool calls that run on GitHub Actions. If Actions workflows are unavailable or fail, Copilot still posts a review, just without those agentic extras. Anthropic's managed Claude Code Review sends several agents through the diff and the surrounding code on its own infrastructure.

Context also comes from outside the code. Qodo documents four sources for its review agents: the codebase and its architecture, pull request history, linked tickets and requirements, and the organisation's standards. Greptile connects to Jira and Linear, and BugBot reads existing pull request comments so it doesn't repeat what a person already said.

Rules and Configuration Files

Rules files are the new hire's team handbook. They are plain text in the repository, and the model reads them as instructions alongside its default review prompt. Each product takes rules through its own files or platform settings, with different scoping.

Where each reviewer reads its rules, taken from each vendor's documentation.

Reviewer	Where Rules Live	How Scope Works
CodeRabbit	.coderabbit.yaml, plus detected guideline files	A guideline covers its own directory
BugBot	.cursor/BUGBOT.md files	Root file plus files above changes
GitHub Copilot	.github/copilot-instructions.md and AGENTS.md	Read from the head branch
Greptile	.greptile/ folders or greptile.json	Child folders override the root
Codex for code review	AGENTS.md Code Review Rules section	Root plus the nearest file
Claude Code for code review	REVIEW.md and CLAUDE.md	REVIEW.md reaches every review agent
Qodo	Review Standards in the Qodo platform	Learned from code and PR history

CodeRabbit's detected guideline files include AGENTS.md, CLAUDE.md, .cursorrules, and .github/copilot-instructions.md, so a rule written for your coding agent can quietly start steering your reviewer too. OpenAI's Codex documentation shows the rule shape that works best across all of them: one outcome to flag, and the safe path.

## Code Review Rules

### Tenant isolation

- Flag any query on customer data that is not
  filtered by the caller's org_id.
  Safe path: pass org_id into the WHERE clause.

Copilot's head-branch behaviour deserves a security reviewer's attention. GitHub's documentation says Copilot reads custom instructions from the branch with your changes, so you can test rule edits in the same pull request. It also means a pull request that edits .github/copilot-instructions.md gets reviewed under its own edited rules.

Static Analysis Alongside the Model

Several of these reviewers are hybrids of rules and models. CodeRabbit documents more than 50 third-party linters and security analysers, such as ESLint and Semgrep, which run in its sandbox and feed their results into the review. It picks tools by the files present, so PHP linters stay off a Java project.

The scanner and model do different jobs. A linter can say with certainty that org_id is unused, and Ruff's ARG001 rule does exactly that when a project enables it, though it has no way to say why that matters. The model can connect the unused argument to the tenant filter in the next function, which is the finding a person needs to act on.

GitHub keeps the two apart more visibly. Its documentation positions GitHub Code Quality, with rules-based CodeQL analysis and optional merge gating, as the complement to Copilot's model-based review. OpenAI's Codex guidance says the same thing from the other side: leave lint and other deterministic checks in CI and out of review rules.

Model Passes and Verification

The model stage is where products differ most, and where vendors say least about exact models. Copilot's documentation says code review uses a tuned mix of models and doesn't support model switching. Its Balanced effort level routes a pull request to a higher-reasoning model, and is the built-in default. Lite aims for faster feedback on common issues.

Multi-pass designs are now common across the category. Anthropic describes Claude Code Review as several agents working in parallel, each hunting a different class of issue, followed by a verification step that checks candidates against actual code behaviour before anything posts. Qodo describes specialised review agents sharing one context layer, and CodeRabbit lists separate review and verification agents.

Filtering is the other half of this stage. Codex posts only P0 and P1 issues on GitHub, so a quiet Codex review may simply mean nothing cleared that high bar. Greptile exposes a strictness setting from 1 to 3, where 1 comments on nearly everything and 3 keeps only critical issues.

Effort settings let you trade money for review depth. On its usage-based plans, BugBot offers Low, Default, High, and a Smart mode that picks a level from your instructions, and Cursor's docs say higher effort can find more bugs at more cost. Greptile offers Base, Plus and Apex review modes with different credit costs. Compare the actual findings before paying for deeper analysis.

Comments, Severity, and Check Runs

Output formatting shapes how the team treats the findings. Claude Code Review tags each finding with a severity, and its Pre-existing tag marks a bug the pull request didn't introduce. That last category is useful, because it separates this author's regression from old debt the reviewer happened to spot.

Check runs are where teams get surprised. Claude Code Review does not report a failure conclusion, so its findings do not block a merge by themselves. BugBot reports findings as neutral by default, and only reports failure when fail-on-unresolved behaviour is enabled.

Even a finished check proves little on its own, because both vendors also report neutral when a run is cancelled or hits an error, and Anthropic's check does the same after a timeout with no findings posted. Read the check's title and summary before you treat it as a clean review.

The approval side of review is shifting too. Copilot's reviews don't count toward required approvals by default, but GitHub now has a public preview in which Copilot can submit an approving review that satisfies the rule. It only applies once admins enable it at every settings level GitHub lists, and GitHub dismisses that approval when new commits arrive.

Learning from Feedback

This is the new hire learning which comments landed. Greptile learns from team comments, from replies to its own comments, from thumbs-up and thumbs-down reactions, and from whether a comment got addressed by the final commit. Its docs describe style comments fading for a team that never acts on them.

BugBot generates learned rules from repository activity and accepts @cursor remember followed by a fact on any pull request. CodeRabbit keeps Learnings from your chat interactions with its bot. Anthropic collects thumbs reactions after merge to tune its reviewer, but its documentation says replies don't prompt Claude to respond or change the pull request.

Learning from reactions cuts both ways for a team. A reviewer that suppresses a category because the team ignored it could suppress useful findings in that category. Review learned rules the way you review code, and delete the ones that encode a bad habit.

What Data Does an AI Reviewer Access?

An AI reviewer can access the code, related files and metadata permitted by its integration. Hosted reviewers typically need repository read access; posting comments or fixes requires additional permissions. Before you install one, trace where the code goes and what the reviewer executes on its servers. The answers differ by vendor and by plan.

CodeRabbit's FAQ says code is shared with OpenAI or Anthropic for review purposes only and never used for training. It also says some data persists, including encrypted code caches and the vector index, along with Learnings. Setting knowledge_base.opt_out to true removes stored knowledge-base data, and self-hosted Enterprise customers can opt out of all retention.

Anthropic's managed Code Review isn't available to organisations with Zero Data Retention enabled. Copilot's agentic context runs on GitHub-hosted runners by default, and teams can switch to self-hosted runners. Greptile documents self-hosted deployment on Docker Compose or Kubernetes for teams that need the index on their own servers.

Execution is the risk that teams most often forget. In January 2025, Kudelski Security reported that CodeRabbit ran RuboCop on a pull request's own .rubocop.yml, which could load an attacker's Ruby file outside the sandbox. The researchers pulled environment variables, including API keys and the CodeRabbit GitHub App private key, which they said gave read and write access to about a million repositories.

CodeRabbit confirmed a fix on January 30, 2025, moved RuboCop into its sandbox, and rotated credentials; Kudelski published the details that August. The lesson applies to every hybrid reviewer. Any tool that runs linters or tests on a stranger's pull request is running that stranger's configuration, so treat pull request content as hostile input and check how the vendor isolates it.

Prompt injection is the model-level version of the same problem. A comment or file in the pull request can contain text aimed at the reviewer instead of the human reader. Keep AI review advisory on public repositories and grant the app the narrowest permissions it offers. On sensitive paths, a bot's approval should never stand in for an accountable human.

What Determines AI Code Review Costs?

AI code review costs depend on the billing unit, review depth and number of runs. Seat plans hide per-review cost behind a monthly fee and a rate limit, while usage plans charge per run or per credit, so every push and every effort upgrade shows up on the bill. Prices can change, so check the vendor's current pricing before you budget.

Billing units and figures as each vendor publishes them for its own tool. Confirm current prices on each vendor's pricing page.

Reviewer	How It Bills	Vendor's Published Figure
CodeRabbit	Per developer, with hourly review limits	Essentials $24/dev/mo, billed annually
Greptile	Per active developer, plus credits	$30/seat/mo with 50 review credits
BugBot	Usage per run on current plans	Credits consumed by each run
GitHub Copilot	AI credits plus Actions minutes	$0.05 to $5 credits per review
Claude Code Review	Usage credits outside plan limits	Token-based usage; cost varies by review level
Codex	ChatGPT plan usage for GitHub reviews	Plus from $20/mo; API keys excluded
Qodo	Shared workspace credit pool	Credit prices on Qodo's pricing page

Copilot's range in that table spans two effort levels. GitHub estimates $0.05 to $1 of AI credits for a Lite review and $0.25 to $5 for Balanced, before Actions minutes, and charges an automatic review to the pull request author. Greptile also bills the author, at $1 per credit once they pass their 50 included credits.

For Codex, OpenAI's pricing page says GitHub code review needs a ChatGPT Plus, Pro, Business or Enterprise plan, because API-key access includes no cloud features. Reviews you run locally with Codex count toward general usage limits instead.

Run the math against your own push habits. A team that pushes eight small commits to each pull request can pay for eight reviews on any per-push, usage-billed setting, since each push that triggers a completed review is billed. The head-to-heads listed in the AI code review tools compared hub put these figures side by side.

A Worked Example

Take the invoice function from the diff earlier and walk it through each stage. The code below reproduces the bug in a runnable form, using an in-memory SQLite database with two organisations.

import sqlite3

db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE invoices (id, org_id, total)")
db.executemany(
    "INSERT INTO invoices VALUES (?, ?, ?)",
    [(1, 10, 120.0), (2, 20, 980.0)],
)

def get_invoice(invoice_id, org_id):
    # New in this PR: org_id is passed in but never used
    return db.execute(
        "SELECT id, org_id, total FROM invoices WHERE id = ?",
        (invoice_id,),
    ).fetchone()

# A user in org 10 asks for invoice 2
print(get_invoice(2, org_id=10))

Running it prints (2, 20, 980.0), which is invoice 2 from organisation 20, handed to a user in organisation 10.

Each stage contributes something different when a pull request containing that function arrives.

1. Diff only: the added lines look tidy and parameterised, so a diff-only pass may say nothing at all.
2. Context: retrieval finds list_invoices and other queries that always filter by org_id, which marks the new function as the odd one out.
3. Rules: the tenant isolation rule above tells the model this pattern is a blocking issue in this repository.
4. Static analysis: an unused-argument rule flags org_id, which points the model to the unused argument.
5. Verification: a second pass confirms the query path is reachable and the argument never reaches SQL.
6. Output: a severe finding lands on the query line, ideally with the fix below as a suggestion.

def get_invoice(invoice_id, org_id):
    # Fixed: the query is scoped to the caller's org
    return db.execute(
        "SELECT id, org_id, total FROM invoices"
        " WHERE id = ? AND org_id = ?",
        (invoice_id, org_id),
    ).fetchone()

print(get_invoice(2, org_id=10))  # another org's invoice
print(get_invoice(1, org_id=10))  # this org's invoice

With the same table setup as before, the fixed version returns nothing for the other organisation's invoice and still finds its own:

None
(1, 10, 120.0)

None of the stages alone makes the finding certain, and several of them lift the odds. That's why two reviewers with the same underlying model can perform very differently on your repository.

What Can AI Code Review Catch and Miss?

AI code review can catch logic errors, missing checks and cross-file inconsistencies when the relevant evidence reaches the model. It can miss defects that depend on unavailable context, and it can still misread code it has seen.

Reviewers do well where the evidence is in text they can read:

• Logic errors inside the changed lines, such as a wrong comparison or a missing null check.
• Changes that break a pattern the index can see, like the missing tenant filter above.
• Security smells with a visible source and sink, such as string-built SQL or a hard-coded secret.
• Violations of rules you wrote down, especially short rules scoped to specific paths.

They miss what never reaches the model. Production configuration, feature flags, live data, and deployment order often live outside the repository, or in files the reviewer never reads. Excluded files are invisible by design, so a Copilot review skips your lockfile entirely. Other repositories only count when the product supports linking them, as CodeRabbit's multi-repo analysis and Greptile's repo clusters do.

Intent is the hardest blind spot to close. A reviewer can't know that a refund limit is a legal requirement unless a ticket or rule says so, which is why Qodo and Greptile pull linked tickets into context. Performance under real load is similar, since reading code rarely reveals how it behaves at ten thousand requests a second.

Finally, a clean review is one sample. GitHub's own documentation says Copilot isn't guaranteed to spot every problem and tells users to validate its feedback. Every vendor in this cluster would say the same about its own tool.

Common Pitfalls and Debugging

Most complaints about AI review trace back to a stage that was left on its defaults. These are the ones that come up most in vendor troubleshooting pages, with the stage that causes each.

The Bot Posts Too Many Comments

Noise usually comes from the filter stage. Raise Greptile's strictness, or cap nits in Claude's REVIEW.md, which Anthropic's example limits to five per review. Then remove rules that duplicate your linter, since a model restating a lint error costs money and adds nothing.

Your Rules Seem to Be Ignored

Check the filename and its location before anything else. CodeRabbit matches guideline filenames case-sensitively, so claude.md doesn't match the **/CLAUDE.md pattern. Listing a guideline file under path_instructions does not load it as review guidance or add it to the review scope. Use the supported guideline configuration to supply its instructions.

BugBot ignores Cursor's .mdc project rules, so put repository rules in .cursor/BUGBOT.md, team rules, or its learned and manual repository rules. It also truncates each rule at 30,000 characters and caps the combined rules at 100,000. Comment bugbot run verbose=true to see exactly which rules a run loaded and which were cut.

The Review Never Ran

Start with the trigger, since most silent reviewers never fired. A manual Codex review needs the exact @codex review text, and automatic Codex reviews only run once Automatic reviews is switched on in Codex settings, and CodeRabbit's plans page says public repositories with fewer than ten stars need a manual trigger.

For Copilot on an organisation licence, the organisation must enable code review in its Copilot policies, and a reached budget blocks reviews along with other credit-consuming features.

The Check Passed with Open Findings

This is the neutral-conclusion behaviour from earlier. If you want merges gated, enable BugBot's fail-on-unresolved option where it's available, or parse the severity counts Anthropic writes into the Claude Code Review check run from your own CI job. Keep a human approval on the branch rule either way.

How Should You Evaluate an AI Reviewer?

Evaluate an AI reviewer by checking whether it finds known defects in your code, how much time its incorrect comments waste, and what each useful review costs. Its access to context matters more than a long feature list.

Architecture trade-offs across the reviewers in this cluster.

Review Approach	What It Does Well	Main Cost
Diff-focused, strict filter	Few, high-priority comments	Misses cross-file breaks
Indexed repository	Catches pattern and caller breaks	Stored index of your code
Linter and model hybrid	Deterministic anchors for findings	Runs untrusted configuration
Multi-agent with verification	Fewer wrong findings posted	Slower and pricier per review

To evaluate a product, test the stages instead of the marketing. The hub's four-week rollout method covers sampling and measurement; add these mechanism checks to it:

1. Replay a past pull request whose bug sat in a file outside the diff, which tests context gathering.
2. Add one rule, then open a pull request that breaks it, which tests rules loading.
3. Reject the same style comment three times and watch whether it fades, which tests learning.
4. Push several small commits and read the bill afterwards, which tests the trigger and cost model.
5. Ask the vendor where the index and learned rules are stored, and how you delete them.

Score only findings that a person has verified. A reviewer that posts two verified bugs a week beats one that posts forty comments you've learned to scroll past.

My Verdict

AI code review is a pipeline wrapped around a model. The trigger sets how often it runs and what you pay, context sets what it can see, rules tell it what matters here, static analysis supplies results from repeatable checks, verification cuts the wrong answers, and feedback shapes the next review.

Those stages give you a useful place to investigate a miss, although the model can still reason incorrectly from complete context.

My verdict, as a security reviewer, is that context and rules matter more than the model name on the box. Pick the reviewer whose context reach matches your codebase, write three sharp rules on day one, and keep its check advisory until your own tests justify letting it block merges.

Then build trust gradually, as you would with a new senior hire, by checking its work on your code.

Frequently Asked Questions

Does an AI code reviewer read my whole repository?

An AI code reviewer may retrieve files from across your repository, but that does not mean every file reaches the model in every review. Repository indexing, file exclusions and the review mode determine what the reviewer can inspect.

Is my code used to train an AI review model?

Whether source code is used to train an AI review model depends on the provider and your agreement. Check the applicable data policy before connecting a repository. Training, temporary processing and retained indexes are separate questions; a no-training promise does not mean no storage.

Why can AI reviews of the same pull request produce different comments?

AI reviews of the same pull request can produce different comments because model outputs vary and each run may retrieve different context. Incremental reviews may also examine only newer commits. Compare verified findings rather than treating a single quiet review as proof that the code is correct.