What work should a business give to AI agents and what should stay with people?

Last reviewed: 4 November 2026 · Chris Paton

What is the KSW test, and why does it matter here?

Knowledge, Skills, Wisdom is a framework I use before I build anything. It comes from how people actually learn, applied to process design. The idea is simple: not all work is the same type, and the type determines whether an agent is a good fit.

Knowledge is facts. Prices, availability, distances, word counts, property attributes. An agent can retrieve and apply these accurately and cheaply.

Skills are repeatable tasks. Syncing a database, generating a draft from a template, running a test suite, sending a triggered email. An agent can execute these faster and more consistently than a person once the process is defined.

Wisdom is judgement. Deciding whether to drop a supplier whose data has started drifting. Choosing which content angle will resonate with a specific audience this month. Knowing when a customer complaint signals a systemic problem rather than a one-off. Wisdom requires context, stakes and consequences. Agents don't have those.

The Toro method starts every automation project with this test. If a task sits cleanly in K or S, it's a candidate for agents. If it sits in W, a person owns it, and an agent at most surfaces the information that person needs to decide faster.

The KSW Test

What does that look like in practice?

Here's how the split plays out across the businesses I run. The TrustedVillas sync pipeline handles 63,000+ properties as at 14 October 2026. Every night it checks availability, updates rates, flags broken feeds and writes structured data back to the database. That is pure K and S: defined inputs, defined rules, verifiable output. No single human could do that volume reliably. And no human should have to.

Campercation's live map holds 25,000+ stopovers as at 14 October 2026. Keeping that data current, geocoded and tagged correctly is an agent job. Deciding which new stopover categories matter to the audience, that's a people job.

The Verqos content engine saves hundreds of hours a week as at 14 October 2026. It generates structured content from verified data inputs. But a person sets the editorial brief, checks the output before it publishes, and decides when the brief needs changing. The agent produces at scale; the person owns quality and direction.

None of these agents are unsupervised. Each one has a human checkpoint before the output reaches a customer or a live system. That's not timidity. That's correct process design.

Which functions should a business start with?

Most businesses that come to me are paying for AI tools and seeing nothing back. The reason is almost always the same: they've tried to automate Wisdom work, or they've skipped the step of cleaning the process first. I ran lean manufacturing before I ran software, and the rule is identical in both: remove waste before you automate. An agent running a broken process runs it faster and breaks more things.

Start with these functions. They score high on K and S, and they're where agents compound fastest.

• Data ingestion and sync. Any pipeline that moves structured data between systems on a schedule. The TrustedVillas property sync is the extreme version. Even a small business has supplier price feeds, inventory counts or CRM updates that an agent handles better than a person.
• Triggered communications. Booking confirmations, follow-up sequences, review requests. The logic is a ruleset. The agent applies it consistently. A person wrote the rules once.
• Monitoring and alerting. Our scheduled agents run 24/7 as at 14 October 2026. They check for broken feeds, missing data, error rates. They don't decide what to do about a problem; they surface it to the person who does.
• Content generation from structured inputs. Product descriptions, location pages, SEO metadata. These work when the input data is clean and the brief is specific. They break when you ask an agent to generate opinion or brand voice from scratch.
• Reporting assembly. Pulling numbers from multiple sources and writing the narrative around them. An agent does the retrieval; a person reads the report and acts on it.

If you're not sure where to start, an AI readiness assessment maps your current processes against K, S and W before you build anything. That's where most of the value comes from: seeing which tasks are actually ready to automate, and which aren't.

Running this split consistently across your operations is how you build a system that compounds. Each week the agents we run surface a problem, save a manual step or extend coverage. That is what the agent log is for. Join the Friday agent log to see what's running, what broke and what we fixed.

What should stay with people, and why?

The Wisdom category is non-negotiable. Here's what it covers in a typical SME.

Supplier and partner relationships. An agent can flag that a supplier's data feed has degraded for three weeks. Only a person decides whether to renegotiate, replace or investigate.

Pricing strategy. An agent can apply a pricing rule. It can't set the rule based on competitive positioning, brand positioning and margin goals that live in a founder's head.

Hiring and team decisions. No comment needed.

Brand and editorial direction. What the business stands for, what it won't publish, how it sounds in a difficult customer moment. These are Wisdom decisions. The agent can produce; a person must approve when it matters.

Escalated customer complaints. Agents handle the first layer well. The moment a complaint involves a legal risk, a loyal customer relationship, or a pattern that signals a product problem, a person takes it.

Anything irreversible. This is the hardest rule to hold. If the agent's output can't be undone, a payment processed, a contract sent, a public statement made, a person signs off. Full stop. The KSW test has a secondary axis: reversibility. High-volume, low-reversibility tasks are exactly the ones where autonomous agents cause the most damage.

How do you apply the KSW test to a task you're unsure about?

Here's the process I use on a whiteboard before any build. It takes about 20 minutes.

1. Write the task as a verb and an object: "Generate property descriptions", "Approve a refund", "Set next month's ad budget".
2. Ask: does this task require new judgement every time, or is it the application of a fixed rule to new data? If the latter, it's K or S.
3. Ask: what's the worst output the agent produces, and what happens before anyone catches it? If the answer is "a customer sees it" or "money moves", add a human checkpoint.
4. Ask: is the underlying process clean? If you can't describe it in five steps, map it first. Don't automate a mess.
5. Ask: can you measure the output? If you can't tell whether the agent is doing a good job, you can't run the agent unsupervised.

If a task passes all five, build the agent. If it fails on steps 3 or 4, clean the process first. If it fails on step 2, it's Wisdom work and the agent supports rather than replaces.

The KSW test is part of a wider set of principles I write about in the Agentic Playbook. If you want a structured way to audit your own processes before you build, that's the right place to start.

Frequently asked questions

What is the KSW test for AI agents?

KSW stands for Knowledge, Skills, Wisdom. Knowledge is factual retrieval. Skills are repeatable, rule-based tasks. Wisdom is contextual judgement that requires understanding stakes and consequences. The test helps a business decide whether a task is a candidate for an AI agent (K or S) or should stay with a person (W). It takes about 20 minutes to apply to a process.

What tasks are best suited to AI agents?

High-volume, rule-bound, reversible tasks: data sync, triggered communications, monitoring and alerting, content generation from structured inputs, and reporting assembly. These score high on Knowledge and Skills in the KSW framework. The key condition is that the output must be verifiable before it reaches a customer or a live system.

What decisions should always stay with people?

Decisions that require Wisdom: supplier and partner strategy, pricing strategy, hiring, brand and editorial direction, escalated customer complaints, and anything irreversible. Reversibility is a secondary test: if an agent's output can't be undone, a payment, a contract, a public statement, a person must approve it before it happens.

How do I know if a process is ready to automate?

Map it first. If you can't describe the process in five steps, it's not clean enough to automate. Check whether the output is measurable, the input data is reliable, and the worst-case error can be caught before it reaches a customer. If the process fails any of those checks, fix the process before you build the agent.

Do AI agents need human oversight?

Yes, always at first, and usually at key checkpoints permanently. Even well-defined K and S tasks need a human checkpoint before output reaches a live system or a customer when the stakes are significant. Agents surface problems; people decide what to do about them. The goal is not to remove people but to let them work on higher-value decisions.

Why do most AI rollouts fail to deliver value?

Two common reasons: automating Wisdom work that agents aren't suited to, and automating a broken process without fixing it first. A flawed workflow run by an agent produces errors faster and at greater scale. The lean principle applies: remove waste before you automate. Most businesses need a process audit before they need an agent.