How Do You Test AI-Written Code?

Last reviewed: 13 October 2026 · Chris Paton

Why does AI-written code need a different review process?

It doesn't, and it does. The fundamentals of code review haven't changed: you're checking that the code does what it's supposed to, that it doesn't break anything else, and that a human can maintain it six months from now. What has changed is the rate of production. An agent can write five hundred lines in the time it takes a developer to write fifty. If your review process relies on a slow drip of PRs and a pair of human eyes, it won't cope.

There's a second problem specific to AI output. Agents are very good at writing code that looks right. The syntax is clean, the naming is sensible, the structure follows the patterns in your codebase. But the logic can be subtly wrong in ways that only surface under edge cases or load. A junior developer who doesn't understand something will leave an obvious gap. An agent that doesn't understand something will paper over it confidently.

That's why ai code review needs both automated gates and deliberate human attention on the logic, not just the style.

What should you do before you prompt the agent?

This is where most teams get it wrong. They write a prompt, get code back, and then try to figure out if it's correct. That's backwards.

Before you send a single prompt, you need three things in place. First, a clear spec: what does this function or feature actually need to do, including the edge cases and the failure modes? Second, tests that define that spec in executable form. Third, type definitions or interface contracts if your language supports them.

This is what I call strategic programming: the human designs the architecture and defines the contract, and the agent fills in the implementation. The agent is fast and accurate within a well-defined box. It's unreliable when you hand it an open field and ask it to decide the shape of the solution.

At Toro we apply lean before agents to every new build: we remove waste from the process before we automate it. In code terms that means eliminating ambiguous requirements, inconsistent naming and missing test coverage before we let an agent near the codebase. An agent will faithfully reproduce your mess, faster.

What automated checks should run on every agent output?

Treat agent output the way you'd treat any untrusted input: validate it before it goes anywhere near production. Here's the gate I run on Toro's own pipelines:

1. Type checking — run your type checker immediately. TypeScript strict mode, mypy, whatever your stack uses. Agent code often passes types at the call site but creates silent mismatches downstream.
2. Linting — catch style violations, unused imports and obvious anti-patterns automatically. Don't waste a senior engineer's review time on things a linter catches in three seconds.
3. Unit tests — the tests you wrote before prompting. If the agent's code doesn't pass your pre-written tests, it hasn't met the spec. Send it back, not forward.
4. Integration tests — run the wider suite to catch regressions. Agents tend to optimise locally and break things in adjacent modules they weren't shown.
5. Security scanning — static analysis tools that flag common vulnerabilities: SQL injection patterns, hardcoded credentials, insecure deserialization. Agents reproduce whatever patterns exist in their training data, including insecure ones.

Only when all five gates pass does the PR reach a human reviewer. This isn't slowing the process down; it's the thing that lets you move fast without accumulating debt you'll spend six months clearing later.

If you want the full framework we use for structuring this on client codebases, get the Agentic Engineering playbook which covers test architecture, agent prompting patterns and review checklists.

Get the Agentic Playbook to see exactly how we set up these gates before deploying agents on a codebase.

What does a good human review of agent code look like?

A human reviewer reading AI-generated code needs to switch off the part of their brain that trusts confident-looking output. The code will look professional. The question is whether the logic is correct.

I train reviewers to ask four specific questions:

• What happens at the boundary? Empty arrays, null inputs, zero values, maximum values. Agents often handle the happy path perfectly and collapse on edge cases.
• What does this assume about the environment? Agents frequently write code that assumes a specific database structure, a specific API response shape, or a specific concurrency model. If that assumption isn't documented, the next developer will break it.
• Where did it take a shortcut? Look for overly broad try/catch blocks, hardcoded values that should be config, and functions that do three things because the agent didn't split them out.
• Can I explain this to someone else? If the reviewer can't explain the logic, it either isn't correct or it isn't maintainable. Both are problems.

The Knowledge, Skills, Wisdom (KSW) framework is useful here: Knowledge is knowing what the code should do; Skills is being able to read and evaluate it; Wisdom is recognising when it's technically correct but the wrong solution. Agents have knowledge from training data. They don't have wisdom. That's what the senior engineer brings.

How does this work on a real pipeline?

I run this on Campercation, one of Toro's own platforms. Campercation's live searchable map currently lists 25,000+ stopovers as at 14 October 2026. The data pipelines that keep that map current, pull in new submissions, validate location data and flag duplicates were all built using agentic coding sessions, with a senior engineer as the architect.

The process looked like this: define the data schema and validation rules first, write tests that check those rules, then prompt the agent to write the pipeline implementation. Every agent output ran through type checking, linting and the validation test suite before I read a line of it. What I reviewed was logic and error handling, not formatting or basic correctness. That's the right division of labour.

What broke? The agent consistently underestimated the messiness of real user-submitted data. It wrote validators that worked perfectly against clean test fixtures and failed on submissions with inconsistent encoding or missing optional fields. That's not a criticism of the agent; it's a reminder that your tests need to be as messy as your real data. The fix was to expand the test cases, not to stop using agents.

The broader lesson: agents are reliable within the box you define. The senior engineer's job is to define a box that reflects reality, not just the ideal case. You can see more about how we structure this work at Toro's agentic engineering practice.

How do you keep this scalable as agent usage grows?

The answer is the ZPD for teams framework: you push each engineer slightly beyond their current comfort zone, with scaffolding, so they build real judgment about AI output rather than becoming dependent on it. A team that reviews agent code well is one where every engineer understands what the agent is likely to get wrong, not just what to look for on a checklist.

Practically, that means rotating the human review seat. Don't let one senior engineer become the AI code reviewer. Every engineer should read agent diffs regularly. Over time the team builds pattern recognition: this is where agents hallucinate, this is where they produce technically valid but semantically wrong solutions, this is where they choose the wrong abstraction.

Keep a running log of the failures. Not to blame the agent or the reviewer, but because the patterns in those failures tell you where your specs and tests were too loose. Tighten the spec, improve the test, and the agent's output on the next similar task will be better.

If you want a senior engineer to look at how your team is currently handling agent code review and identify the gaps, a codebase review is the right starting point.

Frequently asked questions

How do you review code written by an AI agent?

You run automated gates first: type checking, linting, unit tests and integration tests written before the agent produced anything. Once those pass, a senior engineer reviews the diff with specific focus on edge cases, environmental assumptions, hidden shortcuts and overall maintainability. The reviewer checks logic and correctness, not formatting — the automated tools handle that.

What tests should I write before using an AI coding agent?

Write unit tests that define the contract for what you're asking the agent to build: the inputs, the expected outputs and the failure cases. Include edge cases like null inputs, empty collections and boundary values. These tests serve as the spec for the agent and the acceptance gate for its output. If the agent's code doesn't pass them, it hasn't met the requirement.

Can AI agents introduce security vulnerabilities in code?

Yes. Agents reproduce patterns from their training data, which includes insecure code. Common risks include overly broad SQL queries, hardcoded credentials, missing input sanitisation and insecure handling of authentication tokens. Run a static security analysis tool on every agent output before human review. Don't rely on the reviewer to catch all security issues manually.

Does using AI coding agents mean you need fewer senior engineers?

No. It means each senior engineer can cover more ground. Agents produce code fast, but someone with genuine judgment still needs to define the architecture, write the acceptance tests, review the output and catch the subtle logic errors agents consistently produce. The frame at Toro is seniors multiplied, not staff replaced.

What is strategic programming in the context of AI coding agents?

Strategic programming means the human engineer designs the architecture and defines the contracts — types, interfaces, test cases — and the agent fills in the implementation within that structure. It's the opposite of handing the agent a vague brief and reviewing whatever comes back. The more precisely the human defines the box, the more reliable the agent's output inside it.

What are the most common mistakes AI agents make when writing code?

Agents handle the happy path well and fail on edge cases. They assume clean, well-structured inputs even when real data is messy. They tend to write code that optimises locally and breaks adjacent modules. They use broad error handling that swallows failures silently. And they write code that looks architecturally sound but solves a slightly different problem than the one you actually described.