Minimum Viable Evals for Technical AI Workflows
Tech GuruQuick Answer
Compare the current and candidate workflow on saved tasks with explicit expected outcomes and forbidden actions. Hold fixtures and graders fixed, record the system change, and repeat trials where outputs vary. Grade final state and tool behavior, not the model's success message. Block severe failures, report harness errors separately, and retain failed examples; an average score cannot establish safety or explain a regression.
Eval rule: Grade the observable outcome first, then the tool arguments and trajectory that produced it. Repeat nondeterministic trials, separate harness failures from model failures, and keep the prior system available until the replacement clears its release gates.
The Reader Question
How do I know whether my AI workflow is getting better or worse?
This guide is for engineers and technical operators who have a prompt, RAG flow, coding assistant, or tool-using workflow but no reliable release test. You need examples from real work, a person who can judge the domain, and a reproducible environment. You do not need an evaluation platform before you can write clear tasks and deterministic checks.
Current Methodology Context
Agent evaluations need separate task definitions, trials, graders, transcripts, and final-state checks. Deterministic code graders can check schemas and required tool arguments; model-based grading needs calibration against human labels. Pin the runner and grader configuration with each comparison. Framework features do not validate the dataset or rubric.
Before You Start: Safe Defaults
- Do not let an LLM judge be the only check for high-risk tasks.
- Keep sanitized, authorized cases in permission-controlled version storage with expected traits and forbidden failures; do not commit production secrets or private traces.
- Separate severe fail gates from advisory quality trends.
- Record the necessary prompts, versions, retrieved evidence, tool calls, and outputs with redaction, access controls, and a retention limit.
Reference Model
Build cases from observed failures, define checks before comparing candidates, and inspect the failed trials behind each release decision.
Decision Matrix
| Choice | Best Fit | Watch Point |
|---|---|---|
| Exact assertions | JSON, commands, forbidden strings, required citations | Brittle if overused. |
| Rubric judge | Qualitative answer scoring | Needs calibration and human review. |
| RAG metrics | Retrieval relevance, faithfulness, context precision/recall | Depends on good reference data. |
| Human review | High-risk technical decisions | Slower but necessary for trust. |
What Goes in the Dataset
Use real tasks: diagnose a DNS failure, summarize a firewall change, answer a runbook question, produce JSON from a ticket, explain a config stanza, or find which doc supports a claim. Include expected traits and things the answer must not do.
Metrics That Matter
For technical work, measure source citation support, unsupported claims, valid JSON, exact tool name and arguments, config diff correctness, retrieval quality, latency, token use, and final environment state. A tool-using agent can report success while the database, file, ticket, or device remains unchanged; grade the state, not the narration. Record the trajectory too, because a correct outcome reached through an unauthorized tool or destructive intermediate step must still fail.
Trials, Variance, and Confidence
Run deterministic transforms once only when the system truly is deterministic. For sampled or agentic work, run several trials per important case and report both pass count and failure categories. A change that passes 3 of 3 cases once does not establish a stable rate. Small eval sets are useful for catching large regressions, but they do not support fine-grained claims that one model is a few percentage points better. Increase cases and trials when the release decision depends on smaller differences.
CI Gate Without Drama
Do not block every subjective quality dip. Block severe regressions: missing citations when required, unsafe commands, invalid JSON, wrong device/site, or failure to retrieve the known source. Trend style and helpfulness separately.
A Practical Pilot Scenario
Select one workflow and manually review 20-50 recent traces or outputs. Turn the highest-impact failure categories into unambiguous tasks with expected outcomes, forbidden actions, and fixture state. Run the current and candidate systems against the same frozen inputs and environment; use multiple trials where sampling or tool choice varies.
The pilot produces a versioned dataset, raw traces, per-grader results, infrastructure-error counts, and a paired comparison. It also names the domain expert who adjudicated ambiguous cases. An average score without failed-case examples, trial counts, and harness health is not sufficient release evidence.
Implementation Details
Version prompt, model, sampling settings, retrieval corpus and index, tool schemas, permissions, fixture data, graders, and runner. Keep secrets out of traces and test destructive actions only in an isolated environment. If the candidate fails a severe gate, restore the prior prompt/model/tool bundle; do not tune the evaluator after seeing candidate results without recording that the test changed.
- Review 20-50 real examples and define failure categories before automating scores.
- Write task inputs, fixture state, expected outcomes, required tool arguments, and forbidden actions.
- Use code checks for schemas, exact fields, tests, source IDs, policy rules, and final environment state.
- Add a narrow model-based rubric only for qualities deterministic checks cannot express.
- Calibrate that judge against blinded human labels and inspect false passes and false failures.
- Run baseline and candidate with the same fixture data, harness, and graders; record the deliberately changed prompt, model, retrieval, or tool bundle. Use multiple trials where outputs vary.
- Count timeouts, provider errors, fixture failures, and grader crashes separately from task failures.
- Review regressions and retain protected artifacts. Add known misses to the development regression set, while keeping separate held-out release cases out of prompt tuning; disclose when a case has become a development example.
Evidence To Collect
- Dataset and fixture revision, prompt, model, sampling, retrieval snapshot, tool schema and permissions, grader version, and runner version.
- Raw outputs, transcripts, retrieved chunks, exact tool calls and arguments, final environment state, and per-grader reasons.
- Trial count, pass count, severe-failure count, latency distribution, token/cost totals, and paired baseline-versus-candidate results.
- Infrastructure failures reported separately: rate limits, timeouts, broken fixtures, unavailable tools, and grader exceptions.
- Human calibration labels for model judges, including false-positive and false-negative examples.
Validation Checklist
- Eval set includes real examples, not only toy prompts.
- Severe failure conditions are explicit.
- Model, prompt, retrieval, and tool versions are recorded.
- Report shows pass/fail plus latency/cost where relevant.
- Failures result in prompt, retrieval, data, or workflow changes.
Maintenance Cadence
- On every prompt, model, retrieval, tool, or policy change: run severe gates and the paired regression set.
- Weekly during active development: review raw failures and infrastructure errors, not only aggregate scores.
- Monthly: sample recent production traces with authorization and redaction, then add new failure categories and cases.
- Quarterly: recalibrate model judges against fresh human labels and audit whether test cases have leaked into prompts or training data.
Troubleshooting
| Symptom | Likely Cause | First Check |
|---|---|---|
| Eval passes but users complain | Dataset does not match real work | Add recent failures and edge cases. |
| Judge scores feel random | Rubric vague or model unstable | Use clearer rubric and deterministic checks. |
| RAG answers hallucinate | Retrieval poor or answer not grounded | Measure context precision/recall and require citations. |
Common Mistakes
- Judging quality by one impressive chat session.
- Letting an LLM judge approve unsafe technical output alone.
- Changing retrieval chunking without re-running tests.
- Ignoring latency and cost until the workflow is adopted.
- Never adding real production misses back into the test set.
Related TechGeeks Reading
- AI Workflow Notes: Start Here
- Building a Network Digital Twin Workflow
- Implementing AgenticOps Safely: Human Approval, Audit Trails, and Rollback
What This Does Not Protect or Validate
A passing eval proves only that the recorded system cleared the defined graders on the tested cases and trials. It does not prove safety on unseen inputs, correctness of the graders, absence of correlated failures, stable provider behavior, or statistical superiority outside the measured uncertainty. A model judge can reproduce its own biases and must not be the sole gate for consequential actions.
No original eval suite was executed for this article; the procedure is documentation-backed. Production traces may contain personal data, customer content, credentials, source code, or privileged tool output. Minimize and redact the dataset, define retention and access, respect licenses and contractual restrictions, and use isolated fixtures for write-capable tests. Evals reduce known regressions; they do not replace authorization, review, monitoring, rollback, or incident response.
Practical FAQ
How many eval cases are enough?
Start with 20-50 real cases. Add more as misses appear.
Do I need a framework?
Not at first. A versioned dataset plus code assertions can establish the method. Tools such as promptfoo, Ragas, or Langfuse become useful when repeatable orchestration, trace storage, and comparison justify their operational cost.
Can evals prove AI is safe?
No. They reduce regression and expose known failure modes. High-risk work still needs human review.
References
- https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
- https://github.com/promptfoo/promptfoo/releases
- https://www.promptfoo.dev/docs/configuration/expected-outputs/
- https://langfuse.com/changelog/2026-05-28-code-evaluators
- https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/
- https://hamel.dev/blog/posts/evals-faq/
