Working software is a false finish line when nobody can decide whether the next change is safe to release, respond when production differs from the test set, or operate the system after the builder leaves.
Three artefacts close that gap. Evals provide bounded evidence about behaviour. The runbook turns production signals into prepared action. Handover gives the receiving team the artefacts, access, judgement and authority to operate both.
They are often delivered as separate folders. They work as one chain.
Evals support a decision
An eval is useful when its result changes what happens next. It may hold a release, send an output to review, compare a proposed model change or reveal that the represented cases are too narrow.
OpenAI’s current evaluation guidance recommends task-specific cases drawn from the real distribution, separate evaluation of non-deterministic steps, human calibration where automated grading is used, and evaluation on every change. NIST’s voluntary AI RMF Core similarly calls for documented test sets, metrics and tools, testing before deployment and monitoring during operation.
Those sources support the practice, not one universal eval format. For our work, the smallest useful record contains:
| Field | The operating question it answers |
|---|---|
| Objective | Which decision or property is this evidence for? |
| Input class | Which part of the real distribution is represented? |
| Expected property | What must hold, including refusal where relevant? |
| Result | What happened under the stated test? |
| Rationale | Why does this result pass, fail or require review? |
| Version | Which workflow, prompt, model and dependency state ran? |
| Reviewer | Who accepted the criteria and interpreted the result? |
The record should also state known gaps. A passed eval proves only that the represented cases met the stated criteria under the tested version and conditions. It does not prove that the workflow is safe, reliable or correct for every future input.
The release path touches the evidence
An eval dashboard that nobody consults is not a release control. The evidence needs a defined contact point with the delivery path.
Google’s release engineering practice ties tests and an audit trail to the exact release. NIST’s manage function calls for a decision about whether development or deployment should proceed based on measured outcomes. The local implementation can vary, but the release policy should say what a material failure does: hold, change, return to review or proceed under a named exception.
Our delivery-assurance receipts use the same principle for the process around a change. Each stage emits a receipt bound to that change and chained to the stage before it. A missing, expired or mismatched receipt can deny the merge.
The case makes a narrow claim. The chain proves that the recorded process ran against the bound change. It does not prove that the process was good, that defects were prevented or that the gate is impossible to disable. That limit is the difference between evidence and theatre.
The runbook begins with a signal
A useful runbook starts from what an operator can observe, not from a tour of the architecture. Each entry should connect a signal or reported symptom to a response:
- Detect. Name the signal, threshold, refusal or report that opens the entry.
- Contain. Stop further effect, narrow authority, queue work or switch to the fallback.
- Escalate. Name who decides when the prepared response is not enough.
- Recover. Restore a known state, replay safely where permitted and verify the result.
- Record. Preserve the incident, decision and version context. Add a missing case to the eval set where it can prevent a repeat.
NIST’s current incident-response guidance covers preparation, detection, response and recovery across cybersecurity risk management. It is not an AI runbook template. It supports the response lifecycle while the workflow-specific signals and actions remain the team’s responsibility.
A runbook cannot remove judgement. It can reserve judgement for the point where it is needed, with the relevant state already visible and the immediate containment work already named.
Handover is access plus capability
Repository access is necessary and incomplete. The receiving team needs the route to deploy, controlled access to required systems, telemetry, the eval harness, the runbook, incident authority and practised knowledge.
Google’s description of SRE onboarding combines production-readiness review, documentation, training, exercises, access rights and progressive transfer of operational responsibility. That is Google’s service practice rather than a binding handover standard. It demonstrates why a source-code transfer alone is not the operating transfer.
Our property-investment platform case shows the first-party version. The stack was chosen for the team the client would own. The product and team were built together. At handover, nine engineers and a product designer were in place, with the code, process and delivery responsibility passed to the standing in-house team.
The case does not identify the client, publish dates on that page or attribute revenue and unit economics to our intervention. Handover evidence is about the capability transferred, not about borrowing the client’s business performance.
The three contact points
The operating bundle becomes inspectable when each artefact touches the next decision.
Eval to release
A material case fails. The record identifies the version, expected property and reviewer. The release policy returns hold, change or an explicitly authorised exception. The failure does not become a warning that a busy team learns to ignore.
Signal to runbook
The live workflow detects a refusal, exception or degradation. The runbook names containment, escalation, fallback and recovery. The operator records what happened and adds a missing case to the eval set where appropriate.
Artefacts to owner
The receiving team has the repository, deployment route, controlled access, telemetry, eval harness, runbook and authority to act. They have practised the response and can make the next change without waiting for the delivery team to return.
That last sentence is the acceptance test. A folder of documents can be complete while the handover is not.
Delivery-bundle acceptance checklist
| Area | Evidence present now | A promise is not enough |
|---|---|---|
| Evaluation | Cases, criteria, result, version, reviewer and known gaps | “We will add regression tests later.” |
| Release | Stated pass, hold and exception policy bound to the change | “The team checks the dashboard.” |
| Operation | Signals mapped to containment, escalation and recovery | “Contact the builder.” |
| Access | Repository, deployment, telemetry and controlled system access | “Credentials can be arranged.” |
| Capability | Named owners trained on release, incident and change paths | “Documentation has been shared.” |
| Continuity | The receiving team can run the next change and rerun the evidence | “Support remains available.” |
The checklist does not guarantee future safety or uptime. It distinguishes evidence that exists from work that has only been promised, and it makes the remaining responsibility visible before the engagement closes.
Sources and limits
The joined bundle is Esya’s operating contract. Its underlying practices are supported by OpenAI’s evaluation guidance, NIST’s voluntary AI RMF Core, NIST’s incident-response guidance and Google’s SRE responsibility-transfer practice. None defines this exact three-part pack or certifies a workflow that carries it.
The operating-bundle plate is an Esya explanatory reconstruction. It contains no client eval, runbook, credential or confidential delivery artefact.