Skip to content

Question refinement, native web search, citations, streaming#13

Merged
msitarzewski merged 2 commits intomainfrom
question-refinement
Mar 8, 2026
Merged

Question refinement, native web search, citations, streaming#13
msitarzewski merged 2 commits intomainfrom
question-refinement

Conversation

@msitarzewski
Copy link
Owner

Summary

  • Question refinement: Pre-consensus clarification step — analyze question, ask clarifying questions, enrich with answers, then proceed to consensus. API (/api/refine, /api/enrich), CLI (--refine), and frontend (RefinementPanel with tabbed UI, Skip button). Graceful fallback on failure.
  • Native provider web search: Anthropic, Google, Mistral, OpenAI, and Perplexity use server-side search instead of DDG proxy. Per-provider citation extraction (url, title, snippet) on ModelResponse.citations.
  • Citation persistence: citations_json column on Contribution model with SQLite auto-migration. Thread detail API returns citations. Domain-grouped Sources nav (ConsensusNav + ThreadNav) with nested Disclosure components and P/C/R role badges.
  • Anthropic streaming: send() uses _collect_stream() internally to avoid 10-minute timeout with high max_tokens. All downstream parsing unchanged.
  • Parallel challenge streaming: _stream_challenges() uses asyncio.as_completed() — each challenger result appears in the UI immediately instead of waiting for all to finish.
  • Tools enabled by default: web_search wired through CLI, REST, and WebSocket paths. max_tokens bumped 16384 → 32768 across all handlers.
  • Sidebar UX: New-question button, collapsible desktop/mobile sidebar toggle.

Test plan

  • 1641 Python tests passing
  • 194 Vitest tests passing (1835 total)
  • mypy strict clean
  • ruff clean
  • Frontend builds without errors
  • Manual: submit question with --refine, verify clarifying questions appear
  • Manual: verify citations appear in live consensus and stored thread views
  • Manual: verify challengers stream in one-at-a-time during consensus

🤖 Generated with Claude Code

msitarzewski and others added 2 commits March 8, 2026 07:30
…at fix

- Add pre-consensus question refinement: analyze_question() uses most
  expensive model to detect ambiguity, enrich_question() rewrites with
  user's clarifications. REST endpoints POST /api/refine + /api/enrich.
  Frontend: RefinementPanel with tabbed clarification UI, consensus store
  'refining' status, CLI --refine flag.
- Enable tools by default (ToolsConfig.enabled=True). Wire tool_registry
  through REST /api/ask and WebSocket /ws/ask paths (was CLI-only).
- Fix tool format: each provider now transforms generic {name, description,
  parameters} to its native API format (Anthropic: input_schema, OpenAI/
  Mistral/Perplexity: function wrapper, Google: FunctionDeclaration).
- Sidebar: add new-question button (Heroicons pencil-square) + collapsible
  sidebar toggle. Brand left, icons right. Desktop sidebar hide/show.

1619 Python tests + 194 Vitest tests passing. Build clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Native provider web search: Anthropic/Google/Mistral/OpenAI/Perplexity
  use server-side search instead of DDG proxy, with per-provider citation
  extraction (url, title, snippet)
- Citation persistence: citations_json column on Contribution model,
  SQLite auto-migration, thread detail API returns citations
- Domain-grouped Sources nav: ConsensusNav (live) + ThreadNav (stored)
  group citations by hostname with nested Disclosure, P/C/R role badges
- CitationList shared component for inline citation display
- Anthropic streaming: send() uses _collect_stream() internally to avoid
  10-minute timeout with high max_tokens
- Parallel challenge streaming: _stream_challenges() uses
  asyncio.as_completed() to send each result to WebSocket immediately
- max_tokens bumped 16384 → 32768 across all handlers to prevent
  citation truncation
- Memory bank updated with 3 new ADRs and technical documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@msitarzewski msitarzewski merged commit fc09b4c into main Mar 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant