MedGaugeCapabilities / X · Cross-cutting constraints
X.7 Stability Active
Results are reported as pass^3 over three independent runs. Single-run scores are not published.
Evidence: none. No benchmark card lists this capability yet. No level is issued: methodology 0.1.0 sets no level thresholds.
- Domain
- X · Cross-cutting constraints
- Evidence form
- program
- Veto constraints
- —
- Depends on
- —
- Required by
- —
- Internal task families
- —
- External anchors
- —
- Version
- 0.1.0 · 0.1.0
Definition in detail
A clinical tool that succeeds once and fails on the next run cannot be relied on. Reporting only results that hold over repeated runs removes lucky single runs.
Inputs a task gives
- The same task items run three times independently with the same model, harness and benchmark version
- The per-run result records, split by subset (calibration and holdout)
Artifact required
Three result records per item, from independent runs.
How it is graded
A program computes pass^3: an item counts only if it passes in all three runs. Results are reported per subset; single-run scores are not published. Levels are to be thresholds on the claimable lower bound of a passed holdout exam; methodology 0.1.0 sets none and issues no level.
In scope
- Every published result and every level derivation
- Runs with the same model, harness and benchmark version
Out of scope
- Cost per item - X.8
- Improvements to tools measured with pass^3 - graded under B.5
Typical failures
Expected failure patterns; not measured rates.
- Borderline items that pass in some runs and fail in others
- Behaviour that depends on tool call order or tie-breaking
Standards: std-medgauge-capability-framework
Benchmarks
No benchmark card lists this capability.
Grading references
No grading reference is mapped to this capability.