Benchmarks / bm-registry-qc

Specialty Registry QC Planned

A specialty-registry data officer recomputes scale totals and grades from item-level entries, checks missing fields, thresholds and timepoint relations, and logs discrepancies.

Planned. No version of this benchmark is frozen. Nothing on this card is a result. See the roadmap.

care_journey — cross-domain task family. Filed under this card as a source group, but it spans domains: it checks consistency across repeated visits and follow-up due dates (D.7, D.14). Results are still reported at the level of this card.

Capabilities: D.3 Disease registry quality control D.13 Grade and score recalculation D.14 Follow-up target and due-date audit
Source groups: registry-score-recalculation, care-journey-consistency
Task types: Detect and log
External anchors: MedCalc-Bench

Versions

No frozen version.

Task form and materials

Detect-and-log over registry extracts; derived scores are recomputed and compared with recorded values.

Registry extract with item-level scale entries
Data dictionary with scoring definitions

Materials are described by type only.

Artifact fields

Field	Type	Meaning
discrepancies[].record_ref	string	Registry record and field
discrepancies[].kind	enum	One of the defect classes below
discrepancies[].recorded	string	Value as recorded
discrepancies[].recomputed	string or null	Value recomputed from item-level entries, where applicable

Defect and verdict classes

The classes this family covers, at method level. They are not a list of tasks, and a task may contain several classes or none (no-defect controls).

Class	Description
Score does not recompute score_recalculation	A total or grade differs from the value recomputed from item-level entries.
Missing item missing_item	A required item is empty, so the total cannot be valid.
Threshold or target threshold_or_target	A value crosses a defined threshold or misses a follow-up target without the required entry.
Time-point relation timepoint_relation	Follow-up dates or due dates are inconsistent with the protocol schedule.

Illustration of the task shape

Synthetic illustration of the shape only. It is not taken from any task.

Inputs (synthetic)

registry.csv   patient_key, visit, item_1 ... item_n, total, grade
dictionary.md  scale definition, scoring rules, follow-up schedule

Artifact (synthetic)

{"discrepancies": [{"record_ref": "P-41/visit-2/total", "kind": "score_recalculation",
                    "recorded": "14", "recomputed": "12"}]}

Scoring

Form: Program-graded end state
Pass rule: All seeded discrepancies logged, recomputed scores match reference implementation, zero over-reporting, zero corrected values.
Vetoes: X.1 No fabricated evidence X.2 Read-only input integrity X.3 No corrected values X.4 Embedded instruction resistance
Reference-level veto: Yes — a violated veto-flagged grading reference linked to a task fails it
Stability: pass@3 and pass^3 reported together.
Difficulty protocol: Same entry criteria as all MedGauge families.

Scoring details

Recomputation follows the scale's original definition, implemented once as a reference; missing items are logged, never scored as zero.
Writing the recomputed value into the registry is a veto (X.3); the artifact records both values.

Integrity

Gate results, expert key cards and review records are held in the private grading repository. A row is shown here only when its record reference is published.

Contamination control

Holdout answers will not be distributed
One container per task
network disabled

Retirement

A task that enters any training set or targeted debugging is retired and replaced in the next version.

De-identification

Program-generated, or de-identified by removing identifiers in the 18 HIPAA Safe Harbor categories and shifting dates by a consistent per-patient offset (not the Safe Harbor method as such; backed by a re-identification risk assessment).

Grading references

No external clause is cited by this family's checkpoints.

Results

No verified results yetResults appear here only after a site run or re-grade has been reviewed and merged as a result record.

Access

Public

Method
scoring rules
task form with synthetic illustration

Closed

Registry extracts
answers
environment instances
run trajectories

How to test: /rounds/ (applications not open yet).

Cite

MedGauge. Specialty Registry QC (bm-registry-qc). https://medgauge.org/benchmarks/bm-registry-qc/