How to Choose a First AI Pilot: Value-Risk Matrix
An evidence-based process-selection worksheet
Value, readiness, reversibility and operational ownership
Original PILOT-8 matrix and downloadable Markdown checklist
First AI pilot selection, process prioritisation, readiness gates and controlled rollout
$ score candidates --matrix PILOT-8
> value: outcome / frequency / evaluation / owner
> risk: data / integration / reversibility / containment
> route: pilot / redesign / technical-spike / rejectChoosing a first AI pilot is a portfolio decision, not a model-selection exercise. The useful question is not “where could a model be added?” but which bounded process slice can create observable value while keeping uncertainty, integration effort and failure impact under control.
This guide provides an operational worksheet for that decision. It supports the broader AI automation service and AI specialist in Armenia pages with a narrow selection method; it does not replace their commercial intent.
When to use this checklist
Use the matrix after mapping the current process and before choosing a vendor, model or automation platform. It works best when a team has three to seven candidate workflows and can name an owner for each. Do not use it to justify an initiative that has already been politically selected: scoring only helps when the team is willing to reject, redesign or postpone a candidate.
A candidate should be a process slice with a visible start and finish. “Improve customer service” is too broad. “Draft a reply for tier-one delivery-status questions, with an operator approving every send” is scoreable. The slice should identify its user, input, output, system of record and possible side effect.
PILOT-8: criteria and scoring scale
Score every criterion from 0 to 3 using evidence, not enthusiasm. Zero means the condition is absent or unknown; three means it is established and can be inspected.
| Criterion | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| Business value | no named outcome | plausible benefit | baseline and target exist | owner can verify the result |
| Frequency | rare or unknown | monthly | weekly | daily or high-volume |
| Data readiness | inaccessible or prohibited | mostly manual | representative samples exist | access, rights and quality checked |
| Evaluation clarity | “looks good” | subjective review | acceptance examples exist | repeatable pass/fail set exists |
| Integration fit | unknown systems | manual copy required | one stable interface | bounded read/write contract |
| Reversibility | irreversible effect | expensive correction | human approval or correction | shadow/draft mode and rollback |
| Failure impact | unacceptable | high and poorly contained | bounded by controls | low-impact slice with clear stop |
| Ownership | no accountable owner | informal sponsor | process and technical owners named | operation, incident and stop rights agreed |
Separate the result into two axes:
- Value score: business value + frequency + evaluation clarity + ownership.
- Controlled-risk score: data readiness + integration fit + reversibility + failure containment.
Each axis has a maximum of 12. The numbers are not universal benchmarks; they force the same conversation across candidates. Preserve the notes and evidence behind every score because an unsupported “3” is not stronger than an honest “unknown”.
How to complete the matrix
First, describe the current workflow without AI. Record volume, handling time, error or rework pattern, queue delay and the person responsible for the outcome. These are baseline observations, not promised gains.
Second, define the smallest useful AI-assisted change. Prefer classification, extraction, search, summarisation or drafting before autonomous external action. State exactly what remains deterministic and what remains with a person.
Third, score the eight criteria with links or references to evidence: sample cases, API documentation, permission rules, accepted outputs and incident history. If evidence is missing, score the criterion as unknown and create a discovery action. Do not average uncertainty away.
Fourth, define a pilot contract:
- one process owner and one technical owner;
- representative input set and explicit exclusions;
- baseline and acceptance criteria;
- shadow, draft or approval mode;
- maximum volume and spend;
- stop conditions and incident route;
- rollback or manual fallback;
- decision date and evidence required for continue, redesign or stop.
How to interpret the result
High value, controlled risk is the preferred first-pilot zone. The process matters, the evaluation is inspectable and the team can test without granting broad authority. Start with a time-boxed slice and keep production writes behind approval until evidence supports a wider route.
High value, uncontrolled risk is not a green light. Split the workflow, remove irreversible actions, establish data rights, add an evaluation set or build a read-only proof first. The opportunity may be good, but it is not ready to be the first pilot.
Low value, controlled risk can be useful as a technical spike, but should not be presented as business validation. Use it only when it cheaply resolves an architectural unknown that blocks a higher-value candidate.
Low value, uncontrolled risk should be rejected or redesigned. A fashionable model does not repair weak ownership, unavailable data or an unmeasurable outcome.
As a practical gate, a first pilot should have no zero in ownership, reversibility, failure impact or evaluation clarity. It should also have a named baseline and a safe execution mode. This is a decision rule for the worksheet, not a claim that a certain score guarantees ROI.
Common scoring mistakes
Teams often score the theoretical end state instead of the first slice. “The system will eventually resolve every request” hides today’s permissions, exceptions and correction cost. Score only what the pilot will actually read, infer and change.
Another mistake is treating model accuracy as the whole value case. A pilot can produce technically good output and still fail because the queue is too small, integration is expensive, reviewers cannot act on the result or the system adds more reconciliation work than it removes.
Avoid double-counting the same assumption. Faster handling is not automatically more revenue; fewer errors are not automatically lower cost. State the causal path and the observation that would confirm it.
Finally, do not choose a candidate only because its data is easy. Accessible data matters, but the process must have an owner and a decision that changes if the output is good.
What to do after the check
Download the PILOT-8 Markdown worksheet, complete it for three candidate process slices and compare them in one review. Select one bounded candidate, document the reasons for rejecting the others and schedule the next decision before implementation begins.
Then run a short discovery step: validate source access, build the representative case set, confirm integration contracts and test the manual fallback. The first implementation should generate evidence in shadow or draft mode. Only after the acceptance set, operational owner and failure controls work should the team consider a broader rollout.
For implementation options, see AI automation. For inspectable delivery examples, use case studies. A useful first pilot is not the most impressive demo; it is the smallest decision that can produce trustworthy evidence for the next one.
require(processOwner && baseline && acceptanceSet);
require(reversibility > 0 && containment > 0);
route = value >= 8 && controlledRisk >= 8 ? "bounded_pilot" : "redesign_or_reject";