About

About MedGauge

MedGauge is a public benchmark and registry for medical AI agents: benchmark cards, a capability registry, a harness registry, a tool registry, a standards registry, grading references and site-verified results, all published as data with schemas.

The manifesto states what the initiative believes and what it promises.

Principles

• Closed set, open method.
• Artifact tasks graded by program; no judge model in the scored track.
• A result belongs to a model inside a harness.
• Correctness is decided by program against the task rules and specifications supplied with each task; an external clause a frozen version relies on is registered as a grading reference.
• Every number on the site has a source: a registry record built at a known commit, or a linked external page.
• Safety constraints are reported on their own and never averaged into a total.
• MedGauge is independent of the parties it evaluates and of any supplier of evaluation material. Tasks are built, held and graded by the site.

Status

Registries and methodology are published. The first holdout round is in preparation; applications and submissions are not open.

Licences

Code

Apache License 2.0

Content and registries

Creative Commons Attribution 4.0 International (CC BY 4.0)

Datasets

Per dataset card and access tier

Tasks and answers

Not published

Cite

MedGauge: a closed-set, open-method benchmark and harness registry for medical AI agents. https://medgauge.org (registry commit a6e7f26).

Contact

General and corrections

[email protected]

Contributions

By e-mail to the same address; see Contribute

Security disclosure

[email protected] (also in security.txt)

To report a wrong citation, standard or card, name the page and the field. Corrections are listed in the changelog.

Privacy

The site sets no cookies and runs no scripts. The host keeps standard access logs (address, time, requested page). Mail sent to the addresses above is read by the maintainers and is not published.