Form an evaluation
Merion treats evaluation engineering as a governed compilation process—not as automatic trace conversion.1. Register evidence governance
Every episode must reference an active data-use policy that permits evaluation. Register the policy before analysis:2. Analyze a cohort
The cohort records the sampling frame, inclusion and exclusion rules, schema inventory, coverage gaps, and proposed clusters.3. Qualify a task candidate
Qualification explains economic importance, recurrence, capability uncertainty, representativeness, reproducibility, verifiability, non-triviality, rights/privacy, and outcome quality.4. Draft four independent specifications
An evaluation proposal contains separately digest-pinned specifications:TaskSpec: realistic instruction, initial state, allowed actions, and expected end state.HarnessSpec: prompt, tools, permissions, budgets, and stop rules.EvalEnvironmentSpec: reset behavior, fidelity evidence, omissions, dependencies, and secrets boundary.VerifierSpec: capability axes, checks, calibration evidence, exploit hypotheses, and disagreement policy.