Make quality a release decision you can explain.

The handover

What you receive

• A 90-minute intake to review the current evaluator, the workflow and three concrete failures.
• In month one: a versioned evaluation set targeting 100–200 cases where the available evidence supports it. Smaller evidence sets are identified, not padded.
• A judge-to-human agreement check using an existing judge and rubric, plus a CI gate on agreed thresholds.
• A one-page scorecard recording the measurement method, results by failure mode, changes and disagreements, plus a list of published quality claims the evaluation cannot yet substantiate.
• From month two: maintain the case set, compare the judge with fresh accepted labels and rerun the gate against agreed model changes within the monthly time cap.

The starting point

What we need from you

• An existing automated judge with its rubric and threshold, accepted human labels or an agreed labeling path, and a usable CI path.
• Three concrete failures showing the workflow output, the judge’s score and the result your team expected.
• Synthetic or fully redacted fixtures and a technical owner who can approve the release threshold.
• A workflow narrow enough to evaluate in four engineering days initially and two days in each later month.

The decision

How acceptance works

We agree the dataset version, rubric, baseline, human comparison and release thresholds. The scorecard distinguishes observed results, unresolved disagreement and unmeasured behavior; a passing score is limited to that evaluation.

The boundary

What’s outside the scope

• Four engineering days in month one; at most two in each later month. Unused time does not roll over.
• No real personal data, retrieval or reranker tuning, fine-tuning, infrastructure migration or client feature development.
• National-security or law-enforcement work and generic web, dashboard or data-analysis projects are outside this retainer.
• An evaluation service, not a legal certification or a promise that a model is safe in every setting.

Commercial details

Payment & timing

€3,000 per month, invoiced monthly in advance after scope is agreed. Either party can cancel with 30 days’ notice. Any additional work requires a separate agreement; unused days do not roll over.

Applicable taxes, the contracting supplier and payment details are confirmed in the written proposal before payment. Third-party services or usage costs are identified separately.

After delivery

Support with a clear end

Ongoing evaluation work is scheduled within the monthly day allowance. This is not an on-call incident response service. The initial start date and monthly review cadence are agreed together.