[Agentic Phase 1] Integrate agents into find_code pipeline

[bashandbone]

Phase 1: Agent Integration

Parent Epic: #123
Depends On: DI Phase 3 (#119) for agent injection
Target: v0.3
Risk Level: Medium

Integrate pydantic-ai agents into the find_code search pipeline to enable intelligent query understanding and strategy selection.

Goals

• Agent-driven query refinement
• Intelligent search strategy selection
• Tool integration for code-aware reasoning
• Seamless integration with existing search pipeline

Current State

Scaffolded infrastructure in:

• codeweaver.providers.agent - Thin pydantic-ai wrapper
• Registry integration ready

Implementation Checklist

Agent Setup

• Define agent profiles for code search tasks
  • Query understanding agent
  • Strategy selection agent
  • Result synthesis agent
• Configure agent models and parameters
• Implement agent toolsets for code operations

find_code Pipeline Integration

• Add agent injection points in search pipeline
• Implement query preprocessing with agents
• Agent-based strategy selection
  • Semantic vs lexical vs hybrid decisions
  • Chunking strategy selection
  • Filter generation
• Result post-processing with agents
  • Relevance scoring
  • Result explanation
  • Answer synthesis

Toolsets for Code Awareness

• File structure analysis tools
• Code pattern recognition tools
• Symbol lookup tools
• Documentation retrieval tools
• Language-specific reasoning tools

Testing

• Unit tests for agent components
• Integration tests with find_code
• Performance benchmarks (latency impact)
• Quality metrics (search relevance improvement)

Configuration

• Agent enable/disable flags
• Model selection per agent role
• Fallback behavior when agents unavailable
• Cost/latency optimization settings

Success Criteria

• Agents successfully enhance query understanding
• Strategy selection improves result quality
• Performance degradation < 2x baseline
• Tests passing
• Documentation complete
• User-facing agent behavior is transparent

Example Use Cases

1. Natural language query: "Find authentication logic" → Agent identifies security-related patterns
2. Ambiguous query: "login" → Agent determines whether to search for function, UI, or flow
3. Complex query: "How does error handling work?" → Agent orchestrates multi-file analysis

Reference

• Scaffolded code: src/codeweaver/providers/agent/
• Registry: src/codeweaver/common/registry/provider.py

[github-actions]

Claude finished @bashandbone's task —— View job

---

Issue Analysis: Agent Integration Phase 1

I've completed my analysis of this issue and the related codebase infrastructure. Here's my comprehensive review:

---

Summary

This issue proposes integrating pydantic-ai agents into the find_code search pipeline to enable intelligent query understanding and strategy selection. This is Phase 1 of the larger Agentic Enhancements Epic (#123) and depends on DI Phase 3 (#119) for agent injection capabilities.

Constitutional Alignment: ✅ Excellent alignment with CodeWeaver's constitution:

• Principle I (AI-First Context): Agents enhance code understanding and context delivery
• Principle II (Proven Patterns): Leverages pydantic-ai ecosystem patterns
• Principle III (Evidence-Based): Will require benchmarks and quality metrics
• Principle IV (Testing Philosophy): Integration tests with performance validation
• Principle V (Simplicity): Agents improve outcomes without complicating user interface

---

Current State Assessment

✅ What's Already Built

1. Agent Infrastructure Scaffolding (src/codeweaver/providers/agent/):

• Thin wrapper around pydantic-ai with proper re-exports
• Support for 20+ agent providers (OpenAI, Anthropic, Google, Bedrock, etc.)
• Model inference and settings management
• Toolset abstractions (AbstractToolset, CombinedToolset, etc.)

2. Provider Selection System:

• get_agent_model_provider() maps Provider enum to pydantic-ai classes
• infer_agent_provider_class() handles string-based provider inference
• load_default_agent_providers() for dynamic provider discovery

3. Registry Integration (mentioned in issue):

• Provider registry structure exists
• Ready for agent registration

❌ Critical Gaps

1. find_code Pipeline: Currently a stub implementation (src/codeweaver/agent_api/find_code.py:49-68):

async def find_code(...) -> dict:
    """Stub implementation during refactoring."""
    return {
        "matches": [],
        "summary": "find_code tool temporarily disabled during refactoring",
        ...
    }

2. No Agent Integration Points: The search pipeline architecture is commented out (lines 74-248 show the planned flow)

3. Missing Dependencies:

• Issue [DI Phase 4] pydantic-ai integration (agents + data sources) #119 (DI Phase 3): Currently OPEN - provides agent injection infrastructure
• Issue [EPIC] Agentic enhancements: intelligent search, orchestration, and evaluation #123 (Agentic Epic): Currently OPEN - parent epic coordination

---

Implementation Feasibility Analysis

Risk Assessment: MEDIUM-HIGH (Agrees with issue's "Medium" risk level)

Blockers:

1. Dependency Chain: This issue depends on [DI Phase 4] pydantic-ai integration (agents + data sources) #119 (DI Phase 3) which is still open
2. Pipeline State: The core find_code pipeline doesn't exist yet (stub only)
3. No Testing Framework: Limited test infrastructure for integration testing

Complexity Factors:

• Agent integration requires complete pipeline implementation first
• Strategy selection needs semantic/AST infrastructure (partially exists)
• Performance benchmarking requires baseline (doesn't exist with stub)
• "< 2x baseline" performance target is undefined without baseline

Constitutional Compliance Issues

⚠️ Principle III Violation Risk (Evidence-Based Development):

The issue checklist includes:

• "Performance benchmarks (latency impact)"
• "Quality metrics (search relevance improvement)"
• "Performance degradation < 2x baseline"

Problem: Without a working baseline implementation, these metrics cannot be evidence-based. The current stub returns empty results.

Recommendation: Implement working baseline search FIRST (text search at minimum), then add agents with measurable comparison.

---

Recommended Implementation Sequence

Phase 0: Prerequisites (Must complete first)

1. ✅ Complete DI Phase 3 ([DI Phase 4] pydantic-ai integration (agents + data sources) #119) - Provides agent injection
2. ✅ Implement baseline find_code pipeline - Currently stub only
   • Basic text search (code exists in comments, needs integration)
   • File discovery integration
   • Token budgeting
   • Performance baseline metrics

Phase 1A: Foundation (Low-risk agent integration)

3. Define agent profiles and configurations
   • Query understanding agent spec
   • Strategy selection agent spec
   • Result synthesis agent spec
4. Create agent factory functions with DI
5. Build agent toolsets for code operations
6. Unit tests for agent components (isolated)

Phase 1B: Pipeline Integration (Higher-risk integration)

7. Add agent injection points in search pipeline
8. Implement query preprocessing with agents
9. Agent-based strategy selection logic
10. Result post-processing with agents
11. Integration tests with performance comparison

Phase 1C: Validation (Evidence-based evaluation)

12. Performance benchmarks vs. baseline
13. Quality metrics (precision/recall improvements)
14. Cost/latency optimization
15. Documentation and examples

---

Specific Recommendations

1. Update Issue Dependencies

Add explicit dependency on baseline pipeline implementation. The issue should note:

**Prerequisites**:
- [ ] Baseline find_code implementation with text search
- [ ] Performance baseline metrics established
- [ ] DI Phase 3 (#119) completed

2. Agent Profile Specifications

The issue mentions "Define agent profiles" but lacks specifics. Recommend:

Query Understanding Agent:

• Purpose: Parse natural language queries into structured search parameters
• Input: Raw query string + codebase metadata
• Output: Structured query intent + extracted filters
• Model: Fast, cheap model (GPT-3.5-turbo / Claude Haiku level)
• Tools: Language detection, entity extraction, intent classification

Strategy Selection Agent:

• Purpose: Choose optimal search strategy (semantic/lexical/hybrid/AST)
• Input: Structured query + codebase characteristics + performance budgets
• Output: Strategy enum + parameters (weights, chunking approach, filters)
• Model: Medium model with reasoning (GPT-4o-mini / Claude Haiku)
• Tools: Codebase statistics, language capabilities, cost estimation

Result Synthesis Agent:

• Purpose: Rank, filter, and explain results for user's agent
• Input: Raw search results + query context + token budget
• Output: Ranked results + explanations + confidence scores
• Model: High-quality model (GPT-4 / Claude Sonnet level)
• Tools: Relevance scoring, code analysis, explanation generation

3. Performance Budget Allocation

The "< 2x baseline" target needs specificity:

Proposed Latency Budget:

• Query understanding: +100-200ms
• Strategy selection: +50-100ms
• Result synthesis: +200-500ms
• Total agent overhead: 350-800ms
• Baseline target: Must establish first
• Max acceptable: baseline + 800ms (assuming ~400ms baseline = 1200ms total)

4. Toolset Design

The checklist mentions toolsets but lacks detail:

Code Awareness Tools (for agents):

# File structure analysis
@tool
async def get_file_tree(path: Path, max_depth: int = 3) -> dict:
    """Returns project structure for agent context"""
    
# Symbol lookup
@tool  
async def find_symbol(name: str, symbol_type: Literal["function", "class", "variable"]) -> list[Location]:
    """Locates symbol definitions across codebase"""
    
# Language-specific reasoning
@tool
async def analyze_code_pattern(language: str, pattern: str) -> dict:
    """Language-specific code pattern analysis"""

5. Fallback Strategy

Constitutional Principle V (Simplicity) and graceful degradation:

When agents unavailable/failing:

1. Query Understanding fails: Use heuristic NLP (spaCy/regex patterns)
2. Strategy Selection fails: Default to hybrid search with balanced weights
3. Result Synthesis fails: Use rule-based ranking (TF-IDF + recency)

Document this in configuration:

[agents]
enabled = true
fallback_mode = "heuristic"  # Options: "error", "heuristic", "disabled"
timeout_ms = 5000

---

Related Issues & Conflicts

Issue #123 (Agentic Epic)

• Parent epic - coordination needed
• Defines overall vision but Phase 1 is well-scoped
• ✅ No conflicts

Issue #119 (DI Phase 3)

• CRITICAL DEPENDENCY: Agent injection requires DI system
• Currently OPEN with checklist partially complete
• Recommendation: Complete [DI Phase 4] pydantic-ai integration (agents + data sources) #119 first, or merge efforts

Potential Overlap

The commented-out code in find_code.py (lines 81-92) describes a 12-step process:

# 11) Results go to the *context agent* if available for final 
#     filtering and selection, and potential re-querying.

This overlaps with Phase 3 of the epic (Context Agent Tooling #125). Consider:

• Is "Result Synthesis Agent" (this phase) the same as "Context Agent" ([Agentic Phase 2] Integrate data providers into pipeline #125)?
• If different, clarify distinction to avoid duplicate work

---

Estimated Difficulty & Impact

Difficulty: HIGH

• Depends on incomplete baseline implementation
• Requires DI system completion ([DI Phase 4] pydantic-ai integration (agents + data sources) #119)
• Integration complexity with multiple agent roles
• Performance optimization across agent calls
• Evidence-based validation requirements

Impact: HIGH

• Core differentiator for CodeWeaver (AI-first context delivery)
• Enables intelligent search vs. keyword matching
• Foundation for advanced features (Phases 2-5)
• Directly addresses Constitutional Principle I

Time Estimate

• With baseline + DI complete: 2-3 weeks (matches target v0.3)
• Without prerequisites: 4-6 weeks (includes prerequisite work)

---

Suggested Labels

Based on repository label taxonomy:

Primary Labels:

• enhancement - New feature
• pydantic-ai - Uses pydantic-ai agents
• architecture - Significant architectural work
• high-priority - Core differentiator feature

Functional Labels:

• search - Enhances find_code search
• api - Changes agent-facing API behavior
• dependency-injection - Requires DI system ([DI Phase 4] pydantic-ai integration (agents + data sources) #119)

Process Labels:

• blocked - Blocked by [DI Phase 4] pydantic-ai integration (agents + data sources) #119 (or remove if [DI Phase 4] pydantic-ai integration (agents + data sources) #119 completes soon)
• epic - Part of [EPIC] Agentic enhancements: intelligent search, orchestration, and evaluation #123 epic (or use parent linkage)
• research-needed - Agent profile specifications need detail

Quality Labels:

• testing - Comprehensive test requirements
• performance - Performance benchmarking required
• documentation - Extensive documentation needs

---

Recommendations Summary

Immediate Actions

1. ✅ Update issue with explicit prerequisites (baseline + DI Phase 3)
2. ✅ Clarify agent profile specifications (purpose, I/O, models, tools)
3. ✅ Define performance budgets (latency targets per agent)
4. ✅ Specify fallback behaviors (graceful degradation when agents fail)
5. ✅ Coordinate with [Agentic Phase 2] Integrate data providers into pipeline #125 (distinguish Result Synthesis vs. Context Agent)

Before Implementation

1. ⏸️ Complete Issue [DI Phase 4] pydantic-ai integration (agents + data sources) #119 (DI Phase 3) - Critical dependency
2. ⏸️ Implement baseline find_code - Required for evidence-based metrics
3. ⏸️ Establish performance baseline - Needed for "< 2x" target

During Implementation

1. ✅ Follow phased approach (1A → 1B → 1C above)
2. ✅ Evidence-based validation - Metrics at each phase
3. ✅ Constitutional compliance - Verify against all 5 principles

---

Conclusion

This is a well-conceived, strategically important issue that aligns excellently with CodeWeaver's constitutional principles and architectural vision. However, it has critical dependencies that must be resolved first:

1. DI Phase 3 ([DI Phase 4] pydantic-ai integration (agents + data sources) #119) must complete for agent injection
2. Baseline find_code implementation must exist for evidence-based comparison
3. Agent profile specifications need more detail for implementation

Recommendation:

• Label as blocked until [DI Phase 4] pydantic-ai integration (agents + data sources) #119 completes
• Add prerequisite tasks to checklist
• Expand agent profile specifications
• Consider merging baseline implementation work into this issue OR creating a separate "find_code baseline" issue as a prerequisite

This issue represents approximately 2-3 weeks of work (with prerequisites complete) and is appropriately scoped for v0.3 target assuming dependency resolution by early development cycle.

---

Branch: claude/issue-124-20251119-2122