Skip to content

Issue 3: Validator approved a hallucinated answer #4

@sanjaigridsandguides

Description

@sanjaigridsandguides

Problem

  • For FAQ 3 ("Who is the great great great grandfather of Martin Luther King?") the saved answer hedges:

    "...is likely to be Willis Williams... due to the limited availability of historical records, the exact identity ... cannot be confirmed with absolute certainty..."

  • This is unverifiable speculation. The validator passed it with the remark "factually correct, relevant... free from hallucinations."

Impact

  • The validation node literally claims an answer is free of hallucinations while the answer itself admits uncertainty about the fact. The validator is rubber-stamping outputs.

Required Fix

  • Add a rule: "If the answer hedges with phrases like 'likely', 'cannot be confirmed', 'unclear', the answer is treated as low-confidence and should fail validation unless the hedging is essential to the question."
  • Consider using a second pass that asks the validator to cite a source.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions