MedGaugeCapabilities / B · Embodiment (tools)
B.4 Tool result handling Active
Given tool returns that include errors, empty results and timeouts, the agent consumes them correctly and still reaches the required end state. Graded by end-state checks plus error-handling checkpoints.
Evidence: none. No benchmark card lists this capability yet. No level is issued: methodology 0.1.0 sets no level thresholds.
- Domain
- B · Embodiment (tools)
- Evidence form
- checkpoint
- Veto constraints
- —
- Depends on
- B.3 Tool invocation
- Required by
- B.5B.6
- Internal task families
- —
- External anchors
- tau-bench
- Version
- 0.1.0 · 0.1.0
Definition in detail
Real tools fail, return nothing or time out. An agent that reads an empty return as a negative finding, or fills a gap with an invented value, produces confident and wrong artifacts.
Inputs a task gives
- A task and tool set in which some calls return errors, empty results, timeouts or malformed payloads
- A stated retry or fallback policy
- The static materials needed to complete the task once tool issues are handled
Artifact required
The end-state artifact the task requires and, where the task asks for it, an issues log recording tool failures that could not be resolved and the items they affect.
How it is graded
End-state checks confirm the required artifact. Error-handling checkpoints confirm that retries stayed within the stated policy, that empty returns were not reported as negative findings, that no content was invented for failed calls, and that unresolved failures were logged against the affected items.
In scope
- Distinguishing an empty result from a negative result
- Retrying or falling back within the stated policy
- Logging unresolved tool failures and the items they affect
Out of scope
- Choosing which tool to call and with which arguments - graded under B.3
- Key fields missing from the source data itself - graded under X.5
Typical failures
Expected failure patterns; not measured rates.
- Reporting no finding when a lookup returned nothing
- Continuing with stale or partial data after a timeout without noting it
- Retrying without limit
- Writing a plausible value where a call failed
Benchmarks
No benchmark card lists this capability.
Grading references
No grading reference is mapped to this capability.