September 2026
Five Tests To Select AI Pilot For Production
A practical executive test for deciding which AI pilots deserve production funding, what evidence to require, and when to expand, pause or stop.
Executive Snapshot
The Decision
Which AI pilots merit production funding, and how far should each go before organisation commits to wider rollout?
Why Does It Matter?
A pilot can show that AI works in a controlled test. It does not prove that the workflow is used consistently, produces measurable business value, handles exceptions safely or costs less than the work it replaces. Scaling too early multiplies weak results and hidden operating costs; waiting for certainty can leave valuable improvements trapped in experiments.
ETT View
Move selected AI workflows into production in stages. Require a business owner, repeatable demand, reliable performance in real conditions, full operating economics and a funded adoption plan before expanding. Stop pilots that cannot show a credible path to all five.
AI pilots are multiplying faster than proven business outcomes
AI tools make it easier to build and demonstrate pilots, so many organisations now face a more important decision: which experiments deserve the people, process changes and ongoing funding needed for production? A promising result in a short test may come from clean examples, close supervision or unusually motivated users. Those conditions rarely describe everyday work.
Once AI is part of a live process, leaders must account for less predictable cases, changing information, user trust, review effort, service support and the consequences of mistakes. The right test is therefore not whether a pilot can produce a good answer. It is whether a specific workflow can deliver a sustained, measurable improvement while people can use it safely and the full cost remains justified.
"The question isn't whether to invest in AI. It's whether you are ready to scale it."
The Executive Decision Framework
Before committing to wider AI rollout, assess each candidate against five tests: repeated business demand, accountable ownership, safe and reliable operation, full economics, and real adoption. Scale a specific workflow only when it passes all five.
1. The Decision
Question: Does this AI-enabled workflow solve a recurring, material business problem in day-to-day operations?
Why it matters: A successful demo or a handful of impressive examples cannot justify a wider rollout. The case depends on repeated work, enough eligible cases and a measurable improvement over the current process. A tool that is only useful in rare or carefully selected situations may remain a contained experiment.
Evidence to look for: A named workflow and business owner; a baseline for its current time, quality, errors or service level; sustained case volumes; results across ordinary and difficult cases; and evidence that teams will use it repeatedly.
2. Ownership
Question: Is a business leader accountable for the outcome after the pilot team steps away?
Why it matters: In production, someone must own service priorities, the effect of errors, how staff handle exceptions, and whether the workflow is delivering its promised value. Technology delivery alone cannot own a business result.
Evidence to look for: A named executive sponsor and day-to-day process owner; clear responsibility for reviewing AI-assisted work; agreed escalation and fallback steps; an operating budget; and a regular review of outcomes and issues.
3. Control
Question: Does the AI perform safely and reliably with real users, real information and less predictable cases?
Why it matters: A controlled pilot may rely on clean examples and close supervision. Live work includes missing details, unusual situations and changing information. A wrong or confident-sounding answer can create rework or harm if people do not know when to check or override it.
Evidence to look for: Testing against representative routine and difficult cases; clear limits on which information and actions the AI can use; human review for higher-impact work; a way to report and correct poor results; monitoring of errors and changes; and a fallback when the service is unavailable or unreliable.
4. Economics
Question: Does the improved outcome justify the full cost of running and changing this workflow?
Why it matters: Pilot costs can omit integration, data preparation, usage fees, review by staff, support and ongoing improvements. Savings are overstated if people still repeat the old work or spend time correcting AI output.
Evidence to look for: A comparison with the existing process; full recurring and one-time costs; time spent checking and correcting results; cost per completed case; sensitivity to growth in usage; and financial thresholds for expanding, pausing or stopping.
5. Adoption
Question: Will the people doing this work use the AI as part of a better workflow?
Why it matters: Making a tool available does not change how work gets done. If staff do not trust it, lack training or must maintain both old and new processes, the promised benefit may not appear.
Evidence to look for: Named teams and users committed to production use; a redesigned process with clear human responsibilities; practical training and support; repeated use after launch; measures of user acceptance, rework and business outcome; and an explicit plan to retire duplicate steps or tools when evidence supports it.
Scale selectively through business-owned gates
Fund selected AI workflows in stages. Start with named production teams and a clear business owner. Release the next tranche only after the workflow shows repeat use, reliable performance in real conditions, safe handling of exceptions, full costs, and a funded adoption plan. Pause or redesign when usage, outcomes, safeguards, or economics fall short; stop pilots that cannot show a credible path to production value.
Leadership Questions
Which live workflow has improved a business measure, and how do we know AI caused the improvement?
Who owns the result, day-to-day exceptions, user adoption and ongoing cost after the pilot team moves on?
What evidence at the next review would lead us to expand, hold, redesign or stop this use case?
Signal to Watch — Verified value from AI in live workflows
More AI pilots are reaching the point where leaders must decide whether to fund production operation.
Pilot success can be misleading when tests use clean data, close supervision or a small group of enthusiastic users. In live work, exceptions, changing inputs, user habits and support needs shape the actual value.
For each live workflow, track business improvement alongside regular use, correction and exception rates, human review effort, reliability and total cost. Expand only when the benefit is sustained and the full operating burden is understood.
Go Deeper
AI Engineering Beyond the Hype: Harnesses, Guardrails and Production Reality
ETT’s conversation with Adam Witanowski explores why production AI depends on clear success measures, realistic evaluation, secure data access, guardrails, monitoring, cost control and accountable ownership.