Optimize all-agent discovery planning

[tony]

Summary

Closes #40.

This PR makes broad all-agent lookup paths faster by separating metadata-rich source inventory from latency-sensitive search/find discovery. Normal search and find entrypoints now use metadata-free discovery, while source listing and diagnostic inventory surfaces keep version evidence available.

It also pushes coarse search scope into discovery planning: prompt searches enumerate prompt-history stores first and only fall back to conversation stores for agents without prompt-history sources, while conversation scope discovers chat stores directly.

What changed

• Add discovery detail levels: none, catalog, and shape.
• Route CLI/MCP search and find paths through the metadata-free path.
• Keep source listing and inventory APIs on metadata-rich discovery.
• Add role-filtered source discovery so prompt/conversation scope avoids unnecessary store walks.
• Add benchmark definitions for broad all-agent lookup paths and make all capped benchmark names/descriptions include their limits.
• Document fast discovery behavior and scoped discovery planning.
• Declare the benchmark script's direct click dependency for standalone uv run scripts/benchmark.py ... use.

Benchmark evidence

Post-change local timing checks with command output discarded:

grep-all-prompts-json-limit-50          0.731s
grep-all-conversations-json-limit-50    1.816s
search-all-prompts-json-limit-50        0.758s
find-all-prompts-json-limit-500         0.727s

Internal profiling after the planner change:

discover_sources_for_search_prompts     0.309s, 71 sources
run_search_query_prompts                0.513s, 50 records
plan_prompts                            0.068s, 24 sources
collect_prompts                         0.132s, 50 records

For comparison, the issue baseline showed broad all-agent discovery around 2.48s to 3.03s before search planning.

Verification

Ran before the final commit:

$ rm -rf docs/_build; uv run ruff check . --fix --show-fixes; uv run ruff format .; uv run ty check; uv run py.test --reruns 0 -vvv; just build-docs;

Result:

All checks passed
83 files left unchanged
All checks passed
880 passed in 51.13s
build succeeded

Also ran:

$ uv run scripts/benchmark.py list-commands

Confirmed capped all-agent benchmark IDs and descriptions expose their limits.