Answer framing: A useful system needs a defined route for insufficient evidence and uncertainty, with a human who has the time, competence and authority to resolve it.
Why it matters to buyers: NIST’s generative-AI profile identifies confidently presented false content and human over-reliance as risks. It also warns that generated explanations or citations can themselves be false. A fluent rationale is therefore not a substitute for checking the underlying evidence. (nvlpubs.nist.gov)
Put this to: Gromowski, the supplier’s technical lead and the carrier QA leader.
A good answer should demonstrate:
- What happens when a measurement is missing or photographs do not support a proposed item.
- How the interface separates observed facts, derived quantities and unresolved assumptions.
- Where a reviewer sees the original source—not just an AI summary of it.
- Who can override, return or halt the output, and how the reason is recorded.
- How unsupported claims and fabricated evidence references are tested.
- Whether uncertainty thresholds were validated on relevant file types rather than simply selected by the vendor.
Request a controlled test containing a missing dimension, contradictory note and irrelevant attachment. The desired outcome is not necessarily a completed estimate; it may be a correctly bounded draft with a precise request for further evidence.
Sharp follow-up: “What prevents your ‘human in the loop’ from becoming an approval click on a queue they cannot realistically inspect?”
Artifact to request: escalation examples, reviewer-screen demonstrations and evidence of actual review effort.
Expert quote: [Reserved for approved quotation, contributor, role and interview date.]
Sources cited on this page
- nvlpubs.nist.govnvlpubs.nist.gov