MedGauge Charter

Version 0.1.0 · 2026-10-29

1. Name and nature

MedGauge is an independent measurement initiative for medical artificial intelligence. It operates as the GitHub organisation medgauge, the website medgauge.org and a private grading repository. It is not an institute, a society, an accreditation body, a certification scheme or a regulator, and it issues no certificate.

The MedGauge Methodology (docs/METHODOLOGY.md, version 0.1.0) is the normative text of the initiative. This charter binds the initiative and defers to the methodology on every question of measurement, editing, operation and change.

2. Purpose

1. To measure medical AI by the work it produces, under the methodology: artifact tasks graded by program against explicit, sourced rules, with the harness named on every record.
2. To maintain the registries—capabilities, harnesses, tools, standards, benchmark cards, result records, dataset cards, models and grading references—as a reference work with an entry standard, sourcing rules and a review cycle.
3. To publish each frozen object with a persistent identifier, so that any result can be cited with its version and date and any calibration task reproduced in another environment.
4. To accept contributions under recorded acceptance procedures without disclosing task content.

3. Principles

• Closed set, open method: task materials, keys, verifiers and trajectories are closed; every rule that decides a grade is public.
• A grade is decided by a program on the artifact’s fields; no model grades another model.
• A result belongs to a model inside a registered harness version.
• Safety constraints are vetoes, reported in their own columns and never averaged into merit.
• Dimensions are reported each on its own and never combined into a total; a run error is a failed run, never re-run and never removed; every published number can be recomputed from the public record set.
• Every published score is computed by the initiative.
• Only public guidelines, labels, classifications and standards decide a grade.
• The producer is never visible to the grader; no identity enters a grade.
• The initiative is independent of the parties it measures and of any supplier of evaluation material. Tasks are built, held and graded by the initiative.

4. Structure

The initiative is operated by its maintainers. Its founding contributors are recorded under the collective name MedGauge Initiative; see the methodology’s contribution statement (section 27). The roles of reference engineer, task builder, expert reviewer, independent acceptor, data steward, results reviewer, reproducer and contributor, and the records each leaves, are defined in Methodology section 20. Standing in each role is earned by recorded acts under Methodology section 21 and is confirmed by the maintainers.

5. Decisions

Every change to the public repository is a pull request that passes the build checks. Approvals are set by Methodology section 22: one maintainer for a registry record; an independent acceptor and one maintainer for a benchmark version freeze; a results reviewer and one maintainer for a result record; two maintainers for a capability change, a change to thresholds, aggregation or difficulty cut points, a change to the methodology text, or a change to this charter. A dispute on any decision is settled by three independent votes, with the outcome and reasons published.

6. Separation of duties

The builder, the experts who set the key and the acceptor of one task are different people. The author of a grading reference builds no task that tests it. A party measured on a benchmark family contributes no holdout task to it. Whoever submits a subject computes no score. Role assignments are recorded in the private grading repository with each acceptance record.

7. Contribution

Contributions are accepted in the four units of Methodology section 23—a grading reference, a check with provenance, a task naming its checks, and a paired run—through the procedure in CONTRIBUTING.md. An answer key or correction for an existing task, a holdout task from a party measured on that family, a score computed outside the initiative, and any material carrying an origin identity or individually identifiable information are never accepted.

8. Licences

Code and schemas: Apache License 2.0 (LICENSE). Content and registries: Creative Commons Attribution 4.0 International (LICENSE-CONTENT). Contributors keep copyright and license their contributions under the same terms. Task material, keys, verifiers and trajectories are not published under any licence.

9. Publication

Each methodology version, frozen benchmark version, round report and repository release receives a persistent identifier; round protocols are registered before the first run; publications are authored under the collective name MedGauge Initiative, with contributors named with their persistent researcher identifier and contributor role where they consent (Methodology section 27).

10. Name and independence

Third parties may cite MedGauge results with their four-tuple (model, harness and version, benchmark version, date). The name implies no endorsement, certification or accreditation. Registered entries and publications carry the initiative’s own name.

11. Amendment

This charter changes by pull request with two maintainer approvals and a version increment (major.minor.patch), recorded in the repository changelog. The methodology changes under its own Part IV.