MedGaugeCapabilities / A · Foundational
A.1 Medical semantic understanding Active
Given a clinical document, the agent extracts structured fields and assigns standard codes, resolving terms, abbreviations, synonyms and institution-specific conventions correctly. Graded by field-by-field pairing against the reference extraction.
Evidence: planned. Only planned benchmark cards list this capability so far; no frozen benchmark version exists for it.
- Domain
- A · Foundational
- Evidence form
- program
- Veto constraints
- —
- Depends on
- —
- Required by
- C.1D.12D.18D.2D.8D.9
- Internal task families
- —
- External anchors
- MedQA, HealthBench
- Version
- 0.1.0 · 0.1.0
Definition in detail
Review, coding and recomputation all start from reading clinical text the way its author intended. A misread abbreviation or local convention propagates silently into every later artifact built on the extraction.
Inputs a task gives
- Free-text clinical documents such as admission notes, discharge summaries and examination reports
- An extraction schema listing the target fields, value formats and allowed code systems
- A local glossary of abbreviations and documentation conventions, where the task supplies one
- Excerpts of the target code systems at the granularity the schema requires
Artifact required
An extraction record with one row per target field: field_name, extracted_value, normalised_term, code_system, code, assertion_status (present, negated, historical, family history) and source_span, the verbatim text the value was read from.
How it is graded
A program pairs each extracted field with the reference extraction. Values are compared after the normalisation the schema declares; codes must match exactly at the required granularity; assertion_status must match. A field the source does not support must be left empty as the schema directs, and a filled value for such a field counts as an error. Every source_span must occur verbatim in the input.
In scope
- Resolving abbreviations, synonyms and eponyms to standard terms
- Applying institution-specific conventions when they are supplied in the task
- Assigning codes from the supplied code systems at the required granularity
- Separating negated, historical and family-history mentions from current findings
Out of scope
- Judging whether codes assigned by someone else are correct - graded under D.9
- Detecting contradictions within or between documents - graded under A.3 and A.4
- Deciding when information became available - graded under A.5
- Writing narrative text to a specification - graded under A.2
Typical failures
Expected failure patterns; not measured rates.
- Expanding an ambiguous abbreviation to its most common meaning when the supplied glossary defines another
- Coding a negated or ruled-out finding as present
- Choosing a parent code when the schema requires a more specific one
- Filling an unsupported field with a plausible default instead of leaving it empty
Standards: std-snomed-ct · std-loinc · std-rxnorm · std-icd-11
Benchmarks
- Clinical Coding Audit Planned
Grading references
No grading reference is mapped to this capability.