feat: reference librarian — knowledge stage + ask() mode#151
Merged
Conversation
New LIBRARIAN specialist: read-only reference desk over the lab's published knowledge — search_published_capsules, list_lab_docs / get_lab_doc (README, FAQ, methodology docs, the generated experiment catalog), plus the result projections. Three closes: FINAL ANSWER (cited, from corpus) | HANDOFF: build reason=... | HANDOFF: impossible. Wired twice: 1. Librarian-first in run() — structural fix for Finding 21 (0/5 unprompted adoption: schema exposure doesn't bind, a stage does). Corpus-answerable goals terminate as "answered" before config_builder spends anything. Librarian failure falls through to build — a broken reference desk never blocks measurement. 2. ask() mode (multi_agent.py --ask) — the user-delegation use case: surface information without running the lab. Build-worthy questions return "needs_experiment"; the CALLER decides whether to spend. Also: lab_docs logic + /v1/catalog/docs endpoints (progressive disclosure: list names/titles -> fetch one, size-capped with stated truncation; no path-traversal surface — names come from the listing). Live smoke: gemini-2.5-flash in ask() answered "are there published capsules measuring Quack?" with four correct capsule ids and the real published overhead numbers (attach 2.7x@100K -> 4.9x@1M, pushdown ~1.9x). Zero execution. Tests: 7 new (read-only desk pinned — no submit/template tools; three closes parse; ask answers/routes without executing; run() answers from corpus and skips build / proceeds on handoff / proceeds on librarian failure); 6 older orchestrator tests updated for the librarian-first sequence. 38 green + 4 library tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LIBRARIAN specialist (read-only reference desk: capsules + docs + catalog + projections; three closes). Librarian-first in run() = structural Finding-21 fix; ask() mode = the probe-the-lab use case (surface information, never execute; build-worthy → needs_experiment, caller decides). Live smoke: cheapest ladder model answered the quack question with 4 correct capsule ids + real published numbers, zero execution.