Production Prompt Engineering: What Remains After the Demo

A demo proves that a model can answer a favorable prompt. A working process must handle ordinary, incomplete and hostile inputs, respect access rights and leave an auditable decision trail. Production prompt engineering therefore starts with an owner and an accepted outcome. The instruction text is one component of the system.

This guide addresses the narrow question of what to check between prototype and release. For the instruction boundary itself, see the system prompt contract. The broader service is described under prompt engineering, while the Armenia specialist guide covers supplier selection.

Start with the business task

Consider a synthetic support scenario. An operator wants draft answers based on internal instructions. In a demo, one question and one current document produce a fluent answer. In production, questions arrive through several channels, documents change, some records are role restricted, and an answer might affect an order. Readiness means an accepted answer with permitted evidence, or a correct refusal routed to a person.

Write down the unit of work before choosing a model: requester, outcome owner, allowed sources, prohibited disclosure, escalation condition and cost of a wrong answer. Separate document reading from CRM writes. They need different permissions and different ways to confirm an outcome. A prompt cannot invent an undefined business process.

Compare solution options

Choose the smallest option that meets acceptance criteria. When an LLM is involved, version the prompt, source contract, output schema and test set together. A model is not required for every stage of the workflow.

The original RELEASE-GATE-7 decision tree

The following process diagram was created for this article. It is a decision aid, not a customer benchmark.

1. Is the accepted outcome and exception owner defined?
   no → pause automation and clarify the process.
2. Can fixed fields, search or a template meet the goal?
   yes → test that simpler option on normal and failure cases.
3. Must the answer synthesize changing documents?
   yes → scope retrieval, require citations and allow insufficient evidence.
4. Is an external action required?
   yes → authorize separately; bound, deduplicate and read back writes.
5. Are representative fixtures and release thresholds defined?
   no → build the test set before release.
6. Are errors, cost and document drift observable?
   no → use a bounded pilot with human review.
7. Does the candidate pass acceptance and recovery checks?
   yes → roll out in stages with rollback; otherwise revise.

A valid JSON object confirms shape, not truth. A citation ID confirms an allowed source, not that it supports the conclusion. Test these separately.

Selection criteria

Compare every option on the same set of requests, including edge cases. Before seeing results, define evidence adequacy, correct refusal, access violations, time to an accepted answer, cost including human review, and recovery after an uncertain write. Assign an owner and a release rule to each criterion. Your own measurements come from your own process; a vendor demo does not supply them.

Prioritize blocking risks first: cross-tenant disclosure, unconfirmed writes and an absent rollback path. Assess wording and speed after those boundaries. If two options meet the same criteria, prefer the smaller operational burden.

Risks and limitations

Stale or forbidden sources. Retrieval must filter by document version and user rights. A stricter prompt cannot repair a faulty access filter. Missing evidence needs an explicit refusal.

Instructions inside data. A retrieved page can tell the model to ignore its rules. Mark it as untrusted data and enforce permissions and tool execution outside the model. The system prompt contract explores this boundary.

Maintenance cost. A model, schema, source or prompt revision can change answers. Keep regression fixtures, versioned logs and a release owner. Count the cost of an accepted answer, including labeling, review, incidents and updates.

Uncertain writes. A CRM timeout does not prove the write failed. Reconcile destination state and attempt ID before retrying. External actions need a separate AI automation design.

Misleading pilot confidence. A few convenient examples show possibility, not failure coverage. Test incomplete questions, conflicting documents, inaccessible sources, invalid output, role changes, tool errors and ambiguous effects.

A practical rollout sequence

1. Describe one process, its accepted result, exception owner and error cost.
2. Record the baseline without an LLM: inputs, source limits, manual steps and delays.
3. Compare template, workflow and LLM on the same fixtures. Set admission criteria before running them.
4. For LLM use, version the prompt, permitted sources, output schema, refusal route and independent action checks.
5. Run a bounded pilot without automatic irreversible writes. Review every wrong answer and refusal.
6. Prepare release operations: versioned monitoring, call budget, review queue, rollback and update owner.
7. Expand scope only after measuring quality and recovery on your own data.

A useful discovery artifact is a process map, test fixtures, an option comparison and one constrained next step. For an independent review, request a short process audit with a redacted input example and your acceptance rule.