Quality

Review checkpoints and curation

Measure task quality, create a golden set, and improve the next version.

A review checkpoint samples claims produced by selected tasks or output Spaces. It can request automated review, human curation cards, or both.

Review(
    name="event-thread-quality",
    tasks=("group-events",),
    sample_size=100,
    curation_sample_size=30,
    provider="openai",
    model="gpt-5",
    reasoning_effort="low",
    trust_reviewer=False,
)

Parameter	Default	Meaning
`name`	required	Stable checkpoint name.
`tasks`	none	Named tasks covered by the checkpoint.
`output_spaces`	none	Output Spaces covered by the checkpoint.
`sample_size`	50	Unique examples sent for automated review.
`curation_sample_size`	0	Cards prepared for a person. A positive number enables human curation.
`provider / model`	none	Automated review model. Omit for curation-only review.
`max_output_tokens`	2048	Per-request response ceiling for automated review.
`include_reason`	true	Whether automated verdicts include compact per-card evidence. Set false to reduce response cost while retaining every label and confidence.
`seed`	0	Stable sampling seed.
`trust_reviewer`	true	Whether automated verdicts may influence suggestions. Set false to measure only.
`also_optimize_for`	none	Additional declared tasks that should learn from the same reviewed truth.

Golden sets

Accepted human decisions form the golden set for the relevant task and Space. A curation card may include optional reasoning. That reasoning travels with the decision and helps future prompt suggestions understand whether the entire result was wrong or only one field was wrong. Read the golden-set and curation guide.

When suggestions run

Finishing a curation deck starts one improvement pass. If another deck finishes while it is running, the new evidence waits and may be combined into the next pass. Opening a report never starts improvement work. Read how suggestions become a successor version.

Admit, do not mutate

Suggestions do not edit a declared version. Create an editable successor, inspect each prompt or parameter suggestion, accept the ones you want, then declare the successor.