When a Script Beats an AI Agent | Canaan

Somebody has quoted you for an AI agent. The proposal describes a system that reads incoming email, works out what each message is, updates a record and replies. It costs more than you expected, and every reason it costs more is a reason you have no way to check.

Here’s the part the proposal skipped. Most processes pitched as agent work aren’t difficult because they need judgement. They’re difficult because nobody has ever written them down. That’s a different problem, and a model doesn’t solve it.

So if you’re trying to work out whether you need an AI agent, the useful question isn’t what an agent can do. It’s whether your process has a shape you could hand to a new employee on a single sheet of paper.

TL;DR

• The decision isn’t about capability. It’s about whether the process has been defined.
• If you can write the rules down, a script wins: cheaper, testable, and it fails the same way twice.
• If you can’t write them down, an agent won’t rescue you. It’ll fail unpredictably instead of predictably.
• Agents earn their cost in one narrow band: genuinely unstructured input, precisely defined goal.
• Every instance of the “agents cost three to five times more” figure we could trace leads back to token counts comparing one model setup against another. None of them has a script on the other side.

The question is not what an AI agent can do

Agents can do more than scripts. That was never in dispute and it isn’t the decision.

The vocabulary is worth fixing first, because the industry uses one word for two things. Anthropic, which sells the models, draws the line cleanly: workflows are systems where LLMs and tools are orchestrated through predefined code paths, and agents are systems where LLMs dynamically direct their own processes and tool usage. A workflow has a shape you decided in advance. An agent decides its own shape while it runs.

By that definition, most of what gets sold as agent work is a workflow with a model call inside it. The largest count we could find comes from a June 2026 preprint that analysed 6,003 published n8n workflows containing LLM components. Planning or agentic execution was the primary task in 16.6% of them. Text generation (31.1%) and information extraction (18.3%) together accounted for roughly half. In 22.7%, the model node had no path to an external action at all.

Three limits belong with that number. The authors are measuring how the templates are wired rather than what the models did when they actually ran, and they say so. It covers public templates on one low-code platform. And it hasn’t been peer reviewed.

McKinsey’s state of AI survey published in November 2025, covering 1,993 organisations with fieldwork run between 25 June and 29 July 2025, reports 62% at least experimenting with agents and 23% scaling them anywhere in the business. Nearly two-thirds haven’t begun scaling AI across the enterprise at all. Experimenting is cheap. Scaling is where the question you’re being asked to answer actually lives.

[CHART: Grouped bar, three values from one McKinsey survey wave (November 2025, n=1,993 organisations): at least experimenting with agents 62%, scaling agents anywhere in the business 23%, reporting enterprise-level EBIT impact from AI 39%. All three share the same respondent base. Label clearly that this is an enterprise survey population and does not describe small businesses. Alt text: a grouped bar chart of one McKinsey survey of 1,993 organisations, showing 62 percent at least experimenting with AI agents, 23 percent scaling agents anywhere in the business, and 39 percent reporting AI-driven EBIT impact at the enterprise level. The population is enterprises, not small businesses.]

What you’re actually buying is non-determinism

The difference between a script and an agent isn’t intelligence. It’s that one of them gives you the same answer twice.

Everything else follows from that. A script that breaks breaks identically: you reproduce it, fix it once, and a regression test stops it coming back. An agent handed the same input may take a different path on Tuesday than it took on Monday, and the fix is a prompt change whose effect on the other ninety cases you can’t see. You’ve traded a maintenance cost you can schedule for one you can’t. The schedulable kind is the kind that arrives on published dates.

The vendor says so too. Anthropic’s engineering guidance notes that agentic systems often trade latency and cost for better task performance, and that their autonomous nature “means higher costs, and the potential for compounding errors”.

The verification burden is where that lands in practice. Google’s DORA programme, surveying nearly 5,000 technology professionals, found roughly 90% of them now using AI at work, with higher adoption associated with an increase in both delivery throughput and delivery instability, and time saved during creation “frequently re-allocated to auditing and verification”. Stack Overflow’s 2025 Developer Survey found 45.2% saying debugging AI-generated code takes them more time and 66% naming “almost right, but not quite” as their top frustration. Both figures come from the 31,476 respondents who answered that question, not from the 49,009 responses in the survey overall. The population is developers, self-selected. The review doesn’t disappear. It moves.

The distance between adoption and result shows up in the business numbers too. The same McKinsey wave puts the share reporting EBIT impact at the enterprise level at 39%. Adoption and impact aren’t the same curve.

The test is whether you can write the rules down

Ask the person who does the job today to write the procedure as if training a replacement who starts Monday. Not a description of the job. Actual rules: what arrives, where it arrives from, what decides each branch, what happens on each branch, and what to do with anything that fits none of them.

What comes back tells you what to build. There are three outcomes and they’re genuinely different problems.

The rules come out clean: write the script

Every input has a named source and a predictable shape. Every decision is a condition somebody could check by hand. The exceptions are a finite list, even if it’s a long one.

Build the script. It wins on four counts at once. It costs less to build. It costs close to nothing per run. It’s testable, because you can assert an exact output for a known input. And when it fails it fails the same way every time, which means one person fixes it once.

A script is still software, and it inherits the same aftercare any live system needs: dependency updates, somewhere to see it fail, and a named person to fix it.

This is also the advice of the company selling the tokens. Anthropic’s guidance is to find the simplest solution possible and only increase complexity when needed, and its January 2026 follow-up is blunter still: “we’ve seen teams invest months building elaborate multi-agent architectures only to discover that improved prompting on a single agent achieved equivalent results”.

The rules won’t come out, and the goal is vague: automate nothing yet

Nobody can say what the right answer looks like. Two people who do the job describe it differently. The exceptions aren’t a list, they’re most of the work.

The problem is the process, not the tooling. An agent here converts an undefined process into an unaccountable one: you still can’t say what should have happened, and now you can’t say what did.

The cheap move is to run it manually for a month with a written log of every decision and the reason for it. That log becomes either the specification for a script or the evidence that the job needs a person. Either result saves you a build.

The rules won’t come out, but the goal is precise: an agent may pay

The input is genuinely unstructured. Free-text email from customers who write however they write. Photographs. Scanned documents. Recorded calls. No amount of procedure-writing turns that into fields.

But the goal is checkable. Was the invoice number extracted correctly, yes or no. Was this complaint routed to the right team, yes or no. The second half matters more than the first. Without a cheap way to tell whether an output is right, you can’t measure the thing, tune it, or put a ceiling on what it costs.

Anthropic describes the fit as “valuable tasks that involve heavy parallelization, information that exceeds single context windows, and interfacing with numerous complex tools”, and sets an explicit economic gate: “for economic viability, multi-agent systems require tasks where the value of the task is high enough to pay for the increased performance.”

[IMAGE: Simple three-branch decision diagram. Single entry question “Can the rules be written down?” branching to: YES to “write the script”, NO + vague goal to “define the process first, automate nothing”, NO + precise checkable goal to “an agent may be justified”. Warm palette matching #0F0E0C and #C8553D. Alt text: a three-branch decision diagram showing that a clean set of written rules leads to a script, an undefined process leads to defining the process before automating anything, and unstructured input with a checkable goal is the only branch that leads to an AI agent.]

Where a language model doesn’t belong at all

Some jobs have no judgement step in them, and inserting one makes the result worse rather than better. Knowing when not to use an LLM is the cheaper half of this decision.

• Arithmetic and money. Totals, taxes, payroll, reconciliation. A model produces a plausible number. A formula produces the correct one. If the answer has exactly one right value, compute it.
• Anything that must be identical every time. Regulated disclosures, contract clauses, safety and dosage text, the French and English versions of the same legal notice. Variation is the failure mode, and variation is what the model is for.
• Lookups against a source of truth. Stock levels, prices, booking availability, opening hours. Query the system that holds the answer rather than asking something to recall it.
• Deduplication, sorting and matching on a key. Exact algorithms for these have existed for decades, they’re free, and they’re right.
• High volume, low value per item, nobody checking. A per-item model cost across ten thousand items with no acceptance test in between is a budget line that grows with your success, and the errors stay invisible until one reaches a customer.

The short form: if the task has a single correct answer that something else can compute, a model is a slower, costlier and less reliable route to it. Reserve the model for the step where the input is messy and a person would otherwise have to read it.

Where the cost multiples came from

Every page that quotes a cost multiple for agents is quoting the same measurement, and that measurement didn’t have a script in it.

What “three to five times more expensive” actually measured

The claim circulates without a source attached. Across the 81 ranking pages in our own search audit, not one page that quotes a cost multiple links a source for it. Follow it back through its restatements and it lands in one place. In June 2025 Anthropic published an engineering write-up of its multi-agent research system, which states that “agents typically use about 4x more tokens than chat interactions, and multi-agent systems use about 15x more tokens as chats”.

Read the baseline. The comparison is an agent against a chat conversation with the same model. There’s no script, no RPA tool and no deterministic automation anywhere in it. It’s also a count of tokens, not a price.

Anthropic published a second multiple in January 2026: “in our testing, multi-agent implementations typically use 3-10x more tokens than single-agent approaches for equivalent tasks”. Same frame again, one model configuration measured against another. That range sits uncomfortably close to the folk “three to five times” figure. Our reading is that it post-dates most of that claim’s circulation, which makes it a coincidence of shape rather than its origin.

The 4x, the 15x and the 3-10x are one measurement frame restated, not three independent findings. Not one of them benchmarks an agent against a deterministic implementation of the same task.

[CHART: Lollipop chart, token multiples relative to a chat interaction: chat 1x, single agent ~4x, multi-agent ~15x. Annotate the baseline explicitly as “a chat turn with the same model, not a script”. Source: Anthropic engineering, June 2025. Alt text: a lollipop chart of token use relative to a chat interaction with the same model, showing chat at 1 times, a single agent at about 4 times and a multi-agent system at about 15 times. The baseline is a chat turn, not a script.]

The one study that varies only the control strategy

There is a controlled comparison, and it isn’t the one people quote. Lwin and Kumar’s “Deterministic vs. LLM-Controlled Orchestration for COBOL-to-Python Modernization” (arXiv, May 2026) ran 382 COBOL programs from the NIST COBOL85 test suite through three models, and made the methodological move that matters: “we hold the language models, prompts, tools, configurations, and source programs constant while varying only the execution control strategy.”

Both arms still call the same models. What varies is whether code or a model decides the sequence, which is the variable every other published comparison leaves uncontrolled.

Deterministic orchestration reduced token consumption by up to 3.5x. On the sequential file I/O category, letting the model control the sequence cost more than $140 per successful translation against roughly $40, in dollars the paper never identifies.

The paper leaves two things unstated. It gives no currency and publishes no per-token price or pricing date. Given the models tested, those are almost certainly US dollars at US list rates, but nothing in the paper says so and they must not be read as Canadian figures. The dollar comparison also covers one of three categories rather than the study as a whole.

The correctness result is what changes the decision. Deterministic orchestration “achieves comparable computational accuracy to LLM-controlled orchestration while improving worst-case robustness and reducing performance variability across runs”. Cheaper, and at least as correct, with better behaviour on its bad days. The paper never states how many repeated runs that variance claim rests on.

Scope limit, stated plainly: this is legacy code translation with explicit validation stages, not invoice routing at a fifteen-person company. It supports one sentence. Where the steps are known, fixing them beats delegating them.

Ask for cost per accepted result

The COBOL paper prices per successful translation, which means failed attempts are already built into the price of the ones that worked. That’s the right unit and worth taking.

Price per million tokens hides everything that determines the bill: retries, long trajectories that end in nothing usable, human review time, escalation to a person, and the cost of a wrong answer that reaches a customer before anybody notices. The same arithmetic applies to anything quoted at you as a monthly figure. Divide the number by what it actually contains, which is the move set out in what you’re actually paying for in a website maintenance plan.

One other paper publishes that arithmetic properly. Kapoor and colleagues’ “AI Agents That Matter”, in TMLR, priced accuracy on 164 HumanEval problems at April 2024 OpenAI list rates, which are published in US dollars: a deliberately simple baseline reached 93.2% accuracy for $2.45, while the LATS agent scaffolding reached 88.0% for $134.50. The authors pre-empt the staleness objection themselves, noting that these ratios “will be stable even if model costs change”. Their simple baselines are still model calls rather than scripts, so read it as elaborate scaffolding against plain model use. The elaborate option lost on both axes.

Who’s answering this question, and what they sell

We ran a search audit on 22 September 2026 across nine agent-decision queries in English and French, classifying each ranking page by whether its publisher sells something that the answer could favour. Across roughly 81 ranking URLs, 89% had a commercial interest in the answer being yes.

Commercial interest isn’t dishonesty, and reading the set, the pages are better than that framing suggests. The two strongest results on the most commercial query in the audit are vendor pages, and both tell readers to use ordinary automation when the rules are predictable. The clearest statement of the argument made here was published by Anthropic, which sells the tokens. The bias isn’t in the analysis.

It’s in which answers a page like that is able to reach. In our reading, none of those pages has a route to ending with “your operations person can do this in an afternoon with a spreadsheet and a scheduled script”, because that sentence has no product attached to it. Every page that tells you it depends is also the page that bills you when it does.

Gartner named the industrial version of this “agent washing”, the rebranding of assistants, chatbots and RPA tools as agentic AI, and predicts more than 40% of agentic AI projects will be cancelled by the end of 2027 on escalating costs, unclear business value and inadequate risk controls.

That citation deserves the same suspicion as everything else here. Gartner’s own page refuses automated retrieval, and every version of these figures we could read is a reprint of the same June 2025 press release, worded almost identically across publications. By the rule applied to every other source in this post, that’s one source, and one we have not opened at origin. The 40% is also a prediction rather than an observation, so no dataset sits behind it. And the companion estimate, that only around 130 of the thousands of vendors claiming agentic products are genuine, carries no published method, sampling frame or vendor list in any version available to us. Take the phrase, not the figures.

We could not find a methodologically stated audit of commercial products marketed as AI agents. We looked for one. The closest thing that exists is the n8n workflow analysis above, and it counts what users build rather than what vendors sell. The industry’s most-quoted claim about how much agent marketing is fake rests on an unpublished estimate.

How a number degrades in one step

A small example, and the two halves of it sit one click apart. The poll behind that prediction ran in January 2025 among 3,412 attendees of one of Gartner’s own webinars: 19% reported significant investment, 42% conservative investment, 8% none at all, and 31% were waiting or unsure. That description is consistent across every reprint of the release.

A trade publication later described it as a poll of “more than 3,400 organizations actively investing in the technology”. The count survived. The sample didn’t. Nearly four in ten of those respondents had invested nothing or were still deciding, and a self-selected webinar audience became a population of committed buyers. One clause turns a heavily biased sample into evidence about the market.

The better-known case is the figure that 95% of generative AI pilots produce no measurable return. If a deck puts that number in front of you, ask what the sample was, what counted as success, and who funded the work. We’re not restating its figures here, because we couldn’t open a primary copy of the report during this research pass, and a number we haven’t read at source isn’t one we’ll pass on.

Most of the pages answering this question for you have something to sell. So does the deck quoting those statistics at you.

If you run a business rather than a platform

Most small businesses have a scripting problem rather than an AI problem, and the adoption data suggests they already sense it.

Intuit’s 2026 AI Impact Report, published 12 May 2026 and covering more than 34,000 business owners across the United States, Canada, the United Kingdom and Australia, found 77% of US businesses using AI regularly, up from 48% in July 2024. Where they use it is the more interesting finding. In Intuit’s own words, “adoption is highest in marketing, admin, and customer service” and “lowest where human judgment matters most”. Treat the 77% as what it is, a US figure from a company that sells AI features inside accounting software. The pattern underneath it is a sound instinct. Owners reach for models where the output is a draft somebody will read anyway, and avoid them where being wrong costs something.

One absence is worth stating rather than papering over. We looked for Canadian or Quebec data on agent adoption and agent costs among small businesses and found nothing citable. Every figure in this post comes from global enterprise surveys, developer populations or academic benchmarks. The one place we do carry Quebec-specific market figures is what a mobile app costs in Quebec, and that’s build pricing rather than agent pricing. A Montreal operator being quoted for an agent is being sold from data collected somewhere else, about organisations that look nothing like theirs. That isn’t an argument against agents. It’s an argument for asking a vendor what they’ve measured in a business your size, and treating a blank answer as information.

The short version

Write the rules down first. Everything else follows from what comes back.

If the rules come out clean, build the script: cheaper, testable, and it breaks the same way twice. If they won’t come out and nobody can say what a right answer looks like, the process is the problem and no tool fixes it. If they won’t come out but the goal is precise and cheap to check, an agent is defensible, and you should still be able to state the cost per accepted result before anyone writes code.

When a cost multiple is quoted at you, ask what was on the other side of the comparison. In almost everything we could trace, it was another model.

Canaan is a software agency in Mississauga, Ontario. We build web applications, e-commerce, mobile apps and workflow automation, in French and English, and a good part of the first conversation is spent working out whether a process needs a model in it at all, including when the answer is that you don’t need this. Describe the process and we’ll tell you what we would build, and what we would not.

Start a project

Sources

• Anthropic, Building effective agents, 19 December 2024. Source of the workflow-versus-agent definition, the “simplest solution possible” guidance, and the cost and compounding-error caveats. Definitional and advisory; it contains no measurement.
• Anthropic, How we built our multi-agent research system, 13 June 2025. Source of the 4x and 15x token multiples and the economic-viability gate. Both multiples measure token consumption relative to a chat interaction with the same model, not relative to a script or any deterministic automation.
• Anthropic / Claude, Building multi-agent systems: when and how to use them, 23 January 2026. Source of the 3-10x multi-agent-versus-single-agent range and the observation about elaborate architectures matched by better prompting on one agent.
• Naing Oo Lwin and Rajesh Kumar, Deterministic vs. LLM-Controlled Orchestration for COBOL-to-Python Modernization, arXiv:2605.09894, 11 May 2026; related DOI 10.1145/3805760.3814891. 382 COBOL programs from the NIST COBOL85 test suite, average ~1,200 lines, across three models. The paper states no currency and publishes no per-token price or pricing date for its $140 and $40 figures; those figures cover the sequential file I/O category only. The number of repeated runs per program is not stated in the accessible text. The ACM proceedings title is not given on the arXiv record and is therefore not named here.
• Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir and Arvind Narayanan, AI Agents That Matter, arXiv July 2024, published in Transactions on Machine Learning Research, 2025. Costs are at April 2024 OpenAI list pricing, which is published in US dollars; the paper states the date but not the currency. The “simple baselines” are model calls, not scripts.
• Yutian Tang, Yuming Zhou and Huaming Chen, An Empirical Study of LLM-Integrated n8n Workflows, arXiv:2606.29116, v1 27 June 2026, v2 11 July 2026. Figures used here are verified against v2. Preprint, not peer reviewed. 6,003 published templates containing LLM components. The authors state that they measure design-level action coupling and do not claim to measure runtime autonomy, and note that private production workflows are excluded. The collection window is not stated in the paper.
• Gartner, Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027, press release, 25 June 2025. Source of the “agent washing” definition, the 40% prediction, the ~130-genuine-vendor estimate, and the January 2025 poll of 3,412 webinar attendees. Gartner’s site refuses automated retrieval, so every version of these items read during this research pass was a trade-press reprint of this one release. Under the clustering rule applied throughout this post, that is a single source and it was not opened at origin. The 40% is a prediction, not a measurement; the vendor estimate carries no published method.
• MarTech, Gartner: 40% of agentic AI projects will fail, making humans indispensable, 29 April 2026. Cited only as an artefact showing how a sampling frame degrades in retelling, not as a source for any Gartner figure.
• McKinsey & Company, The state of AI in 2025: Agents, innovation, and transformation, November 2025. n=1,993 organisations, fieldwork 25 June to 29 July 2025. Source of the 62% experimenting, 23% scaling agents, nearly-two-thirds not scaling AI across the enterprise, and 39% enterprise-level EBIT impact figures. Linked as the dated PDF rather than the rolling landing page, which now serves a later wave with different numbers. An enterprise survey population; it does not describe small businesses.
• Google Cloud / DORA, Balancing the tensions of AI-assisted software development, 10 March 2026, drawing on the State of AI-assisted Software Development 2025 survey of nearly 5,000 technology professionals. Source of the ~90% adoption figure, the throughput-versus-instability finding, and the reallocation of saved time to auditing and verification. The population is technology professionals, not developers specifically.
• Stack Overflow, 2025 Developer Survey: AI. 49,009 responses used in the published results; the 45.2% debugging-time figure and the 66% “almost right, but not quite” frustration come from the frustrations question, answered by 31,476 respondents. Respondents are developers, a self-selected population.
• Intuit, 2026 AI Impact Report, published 12 May 2026. More than 34,000 business owners surveyed across the United States, Canada, the United Kingdom and Australia, alongside anonymised data from more than 5.3 million QuickBooks businesses, developed with economists at the University of Chicago. The 77% figure is US-specific, and Intuit sells AI features in its products.

Method note on the search audit. Nine agent-decision queries in English and French, roughly 81 ranking URLs, collected on 22 September 2026 from a single location on one day. Each page was classified by hand according to whether its publisher sells agent build services, an agent platform, or the underlying models. That is a judgement about incentive, not about the accuracy of any page, and several of the commercial pages give sound advice. Four of the nine queries are broken out in the table above; the other five are not itemised here. Search results vary by location, personalisation and date, and a rerun would not reproduce the URL set exactly.

Not used, and why. The figure that 95% of generative AI pilots produce no measurable return is referenced only as an example of how statistics travel. We could not open a primary copy of the underlying report during this research pass and therefore do not restate its numbers. McKinsey’s “AI high performer” share, at roughly 6%, comes from a later wave of the same survey than the figures used here and was dropped rather than mixed with them under one date. No Canadian or Quebec data on agent adoption or agent cost among small businesses was found at a citable standard. We are not publishing our own build comparison in this post either, because we do not have one documented to the standard applied to every source above; when we do it will carry the sample, the dates and the hourly basis, or it will not go up.

← All writing