Skip to content

As a nabledge user, I want batch error handling classes to appear in QA answers so that I can correctly implement error handling #187

@kiyotis

Description

@kiyotis

Situation

nabledge-test baseline run (20260312-155314) shows qa-004 detection rate at 4/7 (57.1%), the lowest among all scenarios. The three undetected keywords are TransactionAbnormalEnd, ProcessAbnormalEnd, and ResumeDataReader — all specific to batch error handling.

Previous baseline (20260307-144233) had qa-004 at 7/7 (100%), so this is a regression introduced by the full knowledge file regeneration in v0.5.

Root cause hypothesis: the knowledge-creator generation/fix prompts do not sufficiently instruct the model to include error handling class names (TransactionAbnormalEnd, ProcessAbnormalEnd, ResumeDataReader) in the knowledge files for batch error handling sections. As a result, these class names are either missing or buried in the regenerated files, making them unreachable by knowledge search.

Reference: .pr/00169/nabledge-test/report-202603121553.md — 仮説1

Pain

Users asking "バッチのエラーハンドリングはどうすればいいですか?" receive answers that do not mention the key classes (TransactionAbnormalEnd, ProcessAbnormalEnd, ResumeDataReader), making the answer incomplete and potentially misleading.

Benefit

  • Users can get correct and complete batch error handling guidance including the specific exception classes to use

Success Criteria

  • Root cause identified: confirm which knowledge files are missing the target class names
  • knowledge-creator generation or fix prompt updated to ensure these class names appear in the appropriate sections
  • qa-004 detection rate recovers to ≥ 6/7 in a nabledge-test run after the fix
  • Comparison report generated to confirm improvement vs 20260312-155314 baseline

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions