Red Flags When Choosing an AI Contractor
A practical checklist for checking an AI contractor before signing a contract
Risk gates, scoring scale, pass criteria, data checks, integration review and ownership after launch
How this guide supports the broad Armenia AI specialist landing page with long-tail procurement intent
how to check an AI contractor, AI vendor selection Armenia and production AI risk checklist
$ audit ai_contractor --red-flags
> inspect: outcome / data / integrations / approval / ownership
> score: critical / weak / reviewable / strong
> output: reject / audit_first / controlled_pilotChoosing an AI contractor is risky when the discussion starts with models, demos and confident promises before the contractor has tested the workflow, data, integrations, review boundaries and ownership model.
The broad commercial intent belongs to the AI specialist in Armenia landing page. This article is narrower: it gives a practical red-flag checklist for companies that want to evaluate an AI developer, AI studio or AI contractor before signing a contract.
When to use this checklist
Use the checklist before paid discovery, prototype work, RAG implementation, AI automation, AI agent development, LLM workflow design or internal AI tooling. It is most useful when the team already has a business problem but the contractor has not yet proven how the solution will be scoped, checked and maintained.
A red flag does not always mean "do not hire." Sometimes it means the project should start with audit, data mapping or a small prototype instead of a production build. The risk appears when a contractor ignores the flag or turns it into a vague promise.
Use the checklist when you hear:
- "we can connect any AI model" without asking about the workflow;
- a polished demo before data access and permissions are discussed;
- fixed production promises before integrations are mapped;
- no clear answer about human approval, logging or rollback;
- ownership language that ends at "we will deliver the demo."
Criteria and scoring scale
Score each area from 0 to 3.
| Score | Meaning | Practical interpretation |
|---|---|---|
| 0 | Critical red flag | The contractor avoids the topic or gives generic claims |
| 1 | Weak signal | The topic is mentioned, but no artifact or owner is named |
| 2 | Reviewable | The answer can be checked during discovery or prototype |
| 3 | Strong signal | The contractor names evidence, owner, constraint and next action |
The total score is less important than the blocked category. A contractor can sound strong in model selection and still be unsafe if they cannot explain data access, write permissions, evaluation or maintenance.
Red flags to check
| Area | Red flag | Pass signal |
|---|---|---|
| Business outcome | The contractor talks about AI features before naming the workflow | The workflow, user, decision and failure condition are named |
| Scope | The first offer is a full build without audit or staged release | Scope is split into audit, prototype, pilot and production |
| Data | Data sources, owners and access rights are not discussed | Source list, permissions and sensitive fields are mapped |
| Privacy | The contractor cannot say what must not be sent to a model | Excluded fields and third-party boundaries are explicit |
| AI behavior | The answer is "we will prompt it better" for every risk | RAG, rules, tools, code and human review are compared |
| Evaluation | Quality is judged by impressions instead of examples | Test cases, expected outputs and failure examples exist |
| Integrations | CRM/API/webhook work is treated as a small detail | Integration contracts, rate limits and write paths are listed |
| Human approval | The system can trigger business actions without review boundaries | Approval gates and escalation rules are named |
| Security | Prompt injection, bad documents and leakage are ignored | Failure modes and controls are described plainly |
| Delivery | The only promised artifact is a demo | Repo, docs, logs, deployment notes and handoff are included |
| Maintenance | Nobody owns fixes after launch | Support owner, monitoring and change process are visible |
How to fill it in
Run the checklist internally before the contractor call. Then use the same table during the call and ask the contractor to improve the missing parts. A strong contractor should welcome this because it makes the project easier to scope.
For every criterion, write:
- current answer;
- evidence or missing input;
- score from 0 to 3;
- owner;
- next action;
- decision: proceed, audit first, pause or reject.
This makes vendor selection less dependent on confidence and more dependent on observable delivery behavior.
How to interpret the result
Use this routing logic after the checklist.
| Result | Interpretation | Next step |
|---|---|---|
| Any 0 in privacy, writes or rollback | Production delivery is unsafe | Stop production scope and run risk audit |
| Many 1 scores in data or integrations | The demo may not survive real operations | Map data sources and API contracts |
| Weak evaluation answers | Quality will be argued by opinion | Build an evaluation set before implementation |
| Weak ownership answers | The system may become unsupported after launch | Define support, logs and handoff before contract |
| Mostly 2 and 3 scores | The contractor is ready for a scoped next step | Start audit, prototype or controlled pilot |
For production AI, pay special attention to data access, sensitive fields, write permissions, human approval, logs, rollback and maintenance. These are the areas where a beautiful demo can hide the most expensive failure.
Downloadable red-flag template
Use this Markdown template as the original proof artifact.
| Area | Red flag observed? | Evidence | Score 0-3 | Owner | Next action | Decision |
|---|---|---|---|---|---|---|
| Business outcome | ||||||
| Scope staging | ||||||
| Data access | ||||||
| Privacy boundary | ||||||
| Evaluation | ||||||
| Integrations | ||||||
| Human approval | ||||||
| Security failure modes | ||||||
| Delivery artifacts | ||||||
| Maintenance owner |
The full Markdown file is available as a project asset: AI contractor red flags checklist.
What to do after the check
If the checklist shows critical red flags, do not expand scope. Start with an audit, integration map or small proof with explicit stop conditions. If the contractor handles the checklist well, use it to define the first safe delivery milestone.
For broader service context, use the AI specialist in Armenia page. For proof, review the case studies. For neighboring procurement criteria, compare the guides on choosing an AI developer, freelancer vs AI studio and 25 questions before starting an AI project.
Checked and updated
Checked on 2026-07-09 against the aicoding.am content plan, existing service pages, public case-study layer and current local article cluster. The article avoids self-awarded rankings and keeps broad local AI intent with the dedicated landing page.
Where This Applies
AI contractor review, procurement risk gate and first production-safe scope
This article is useful when a company in Armenia wants to evaluate an AI contractor before committing to an AI automation, RAG, agent, LLM workflow or internal tool project.
- Founders checking whether a polished demo hides production risk.
- Operations teams turning vendor claims into a scorable review.
- Companies that need red flags, pass criteria and next actions before a contract.
contractor_risk = unclear_outcome
+ hidden_data_access
+ unmanaged_write_permissions
+ missing_evaluation
+ no_maintenance_owner;
if (score.privacy == 0 || score.rollback == 0) stop("production_scope");
if (score.integrations <= 1) require("integration_map");