Is Your Data Ready for AI Automation? An Express Audit
A process-specific readiness check before model selection
Access, rights, quality, meaning, freshness and operational ownership
Original DATA-5 maturity matrix with defect priorities and decision gates
AI automation data audit, readiness criteria, repair priorities and bounded pilot routing
$ audit data --matrix DATA-5
> inspect: access / rights / quality / meaning / freshness
> classify: P0 stop / P1 repair / P2 control / P3 improve
> route: blocked / discovery / bounded-pilot / readyA data audit for AI automation is not a search for a perfectly clean database. It is a controlled check of whether a specific process has enough accessible, lawful, interpretable and observable data to support a bounded decision. The useful question is not “do we have data?” but can the team trace the inputs, reproduce the expected output and correct a failure before it creates an unacceptable side effect?
This express audit supports the broader AI automation service and AI specialist in Armenia pages with a narrow readiness method. It does not replace their commercial intent and it does not promise that passing a checklist guarantees model quality or ROI.
Audit scope: one process, not the whole company
Start with one process slice and one proposed AI-assisted action. “Audit the CRM” is too broad. “Use incoming request text and the customer record to propose a routing category, while an operator confirms the queue” is inspectable.
Write down six boundaries before scoring:
- the trigger that starts the process;
- the source records and files the system may read;
- the field, document or event that represents the expected outcome;
- the systems and people that change those records;
- the proposed AI output and its consumer;
- the side effect, approval and recovery path.
The audit should use representative examples, not a hand-picked demo set. Include normal cases, rare categories, missing fields, duplicates, mixed languages, delayed updates and known exceptions. If the process has seasonal or operational shifts, sample across those periods.
Do not pull unrelated personal, confidential or regulated fields into the audit “just in case”. Confirm purpose, access rights, retention and deletion rules before copying data into a model, evaluation workspace or vendor platform. Legal requirements depend on the organisation and jurisdiction; this checklist identifies questions but is not legal advice.
DATA-5: the readiness matrix
Score five areas from 0 to 3. Zero means absent, prohibited or unknown. One means usable only through fragile manual work. Two means a bounded pilot is possible with explicit repairs or controls. Three means the condition is documented, testable and owned.
| Audit area | 0 — blocked | 1 — fragile | 2 — pilot-ready with controls | 3 — operationally ready |
|---|---|---|---|---|
| Inventory and access | sources or owner unknown | manual exports and personal access | named sources, owner and bounded export/API | governed access, versioned schema and least privilege |
| Rights and purpose | rights or permitted use unknown | assumptions or inconsistent consent | approved purpose, exclusions and retention rule | auditable policy, deletion route and vendor boundaries |
| Quality and coverage | no representative sample | silent missingness, duplicates or label drift | defects measured by segment; repair plan exists | automated checks, thresholds and exception queue |
| Meaning and lineage | fields lack shared meaning | business rules live in people’s memory | glossary, source-of-truth and joins documented | lineage, versions and change ownership are observable |
| Freshness and operations | update timing unknown | stale snapshots and manual reconciliation | freshness target, fallback and incident owner | monitored lag, replay/idempotency and recovery runbook |
Keep evidence beside every score: query result, schema, policy, sample review, API contract, data-quality check or owner confirmation. A score without evidence is an assumption. Do not average a zero in rights or access into a comfortable total.
The matrix is intentionally process-specific. The same CRM may be ready for read-only search but not for autonomous pricing changes. The same document archive may support retrieval after access filtering but fail an extraction use case because the target fields were never recorded consistently.
Maturity levels and decision gates
Use the lowest critical area to set the route:
- Level 0 — blocked: source, rights, outcome or owner is unknown. Stop implementation and resolve the blocker.
- Level 1 — discovery only: the team can inspect samples but cannot yet produce a stable evaluation or integration contract.
- Level 2 — bounded pilot: representative cases, explicit exclusions and a safe read-only, shadow, draft or approval mode exist.
- Level 3 — operationally ready: checks, ownership, monitoring, correction and change control support a production rollout.
A total score is useful for comparing snapshots, but it must not override critical gates. Do not start a pilot if rights and purpose are zero, if no one owns the source, if the expected output cannot be reconstructed, or if a consequential write has no approval and rollback path.
For a first pilot, a practical threshold is at least level 2 in inventory, quality, meaning and operations, plus explicit approval of rights and purpose. This is a planning rule for DATA-5, not an industry benchmark.
Typical defects and how to detect them
Unknown source of truth. Two exports contain different customer status, product availability or ticket owner values. Detect this by tracing three to five real records from trigger to final outcome and naming the authoritative field at each step.
Silent missingness. A field is populated for easy cases and absent exactly where the process is difficult. Measure missingness by category, language, channel, team and period—not only as a global percentage.
Historical labels that describe old behaviour. A routing or resolution label may reflect a previous policy, a former team structure or shortcuts used to close tickets. Review label definitions and sample disagreements with current operators.
Duplicates and identity collisions. One customer, company or product appears under several identifiers; or two entities share a weak key. Test joins, deduplication rules and merge history before evaluation.
Unrecorded exceptions. Operators resolve edge cases in chat, spreadsheets or memory, while the main system shows only the final state. Map exception paths and include them in the representative set.
Data leakage. The expected answer or a later human correction appears in the input, evaluation split or retrieved context. Separate timestamps, identities and train/test periods so the system cannot “predict” information that would not exist at decision time.
Freshness mismatch. The model reads a nightly export while the operational decision depends on changes from the last few minutes. Record event time, processing time and acceptable lag; define what happens when the freshness target is missed.
Unstable semantics. A field name remains unchanged while its meaning changes. Version taxonomies and business rules, and connect changes to evaluation results and downstream contracts.
Prioritise repairs by risk, not convenience
Classify findings by the decision they block:
| Priority | Meaning | Examples | Required route |
|---|---|---|---|
| P0 — stop | unsafe, prohibited or impossible to verify | unknown rights, exposed secrets, no source owner, irreversible write without recovery | stop; remove access or resolve governance before any model test |
| P1 — repair before pilot | likely to invalidate evaluation or workflow | missing target outcome, broken joins, severe segment gaps, stale source, inconsistent labels | repair, resample and rerun the affected checks |
| P2 — control in pilot | bounded defect with a safe operating response | rare unsupported format, low-volume exception, known manual fallback | exclude or route to review; monitor explicitly |
| P3 — improvement | does not block current acceptance criteria | optional enrichment, convenience field, broader coverage | place in backlog; do not delay evidence |
Fix the smallest cause that changes readiness. A new data platform is not the default answer. Sometimes the correct repair is a documented field definition, an API permission, an exception queue, a timestamp, a representative evaluation set or a decision to narrow the pilot.
After each repair, rerun only the checks that depend on it and record the new evidence. Do not simply change the score. Readiness is a traceable state, not a presentation.
Format of the final audit report
Keep the report short enough to drive a decision. It should contain:
- Scope and proposed action: trigger, input, output, systems, owner and possible side effect.
- Source register: source of truth, access method, schema/version, owner, rights, retention and freshness.
- Representative sample: selection method, segments, periods, exclusions and known blind spots.
- DATA-5 matrix: score, evidence and uncertainty for every area.
- Defect register: finding, affected segment, impact, priority, repair owner and verification step.
- Pilot gate: blocked, discovery only, bounded pilot or operationally ready.
- Control contract: allowed reads/writes, review route, monitoring, stop condition, fallback and rollback.
- Next decision: date, required evidence and the person authorised to continue, redesign or stop.
The report should distinguish observed facts from hypotheses. “18 of 50 sampled requests lacked a verified resolution field” is evidence from a bounded sample. “Cleaning the CRM will reduce handling time by 30%” is an unsupported forecast unless a measured causal test exists.
What to do after the express audit
If the result is blocked, resolve rights, ownership or traceability before testing a model. If it is discovery-only, build the source register and representative case set. If it is ready for a bounded pilot, keep the first run read-only, shadow, draft or approval-based and measure results against the same sample definition.
Connect this check to the first AI pilot value-risk matrix: data readiness is one gate, while value, evaluation, reversibility, integration and ownership determine whether the process is a good first candidate. Review case studies for inspectable delivery patterns.
Good data readiness does not mean “all data is clean”. It means the team knows what the process reads, what the fields mean, which defects matter, who owns correction and how the system stops safely when evidence is insufficient.
require(sourceOwner && permittedPurpose && representativeCases);
require(targetOutcome && freshnessTarget && correctionPath);
route = criticalZero ? "blocked" : minReadiness >= 2 ? "bounded_pilot" : "repair_first";