Implement continuous analysis service (ChainWeaverService daemon)

[dgenio]

Context / Problem

ChainWeaver's backlog decomposes the "compile-away-LLM-calls" vision into library-level building blocks: ChainAnalyzer (#77) discovers valid chains from schemas, ChainObserver (#78) detects runtime patterns, the LLM compiler (#28) proposes flows semantically, the description optimizer (#100) rewrites tool descriptions, and GovernanceManager (#13) gates promotion. But all of these are pull-based APIs — a developer must manually invoke them.

The original product vision describes a continuously-running service (conceptually "offline" relative to agent calls) that:

1. Watches for tool registry changes (new MCP tools, updated schemas)
2. Automatically runs schema compatibility analysis
3. Monitors agent tool call sequences for recurring patterns
4. Proposes compiled flows and description rewrites to a review queue
5. Presents approved flows as tools to the agent LLM — closing the loop

Without this service layer, users must manually orchestrate the analysis → scoring → proposal → governance → registration pipeline. The service transforms ChainWeaver from a library into a continuous optimization layer for MCP agent ecosystems.

Proposal

1. ChainWeaverService — the integration daemon

Create chainweaver/service.py:

class ChainWeaverService:
    """
    Continuous analysis service that ties together the ChainWeaver
    optimization pipeline: analyze → observe → propose → govern → expose.

    Runs as a background process, separate from agent runtime.
    """

    def __init__(
        self,
        *,
        registry: FlowRegistry,
        executor: FlowExecutor,
        analyzer: ChainAnalyzer,          # from #77
        observer: ChainObserver,           # from #78
        governance: GovernanceManager,     # from #13
        llm_fn: Callable[[str], str] | None = None,  # for LLM-assisted proposals
        config: ServiceConfig = ServiceConfig(),
    ) -> None: ...

    def run(self) -> None:
        """Start the continuous analysis loop."""

    def stop(self) -> None:
        """Gracefully stop the service."""

2. ServiceConfig — configurable triggers and thresholds

@dataclass
class ServiceConfig:
    # Analysis triggers
    analyze_on_tool_change: bool = True       # Re-analyze when tools are added/updated
    analyze_interval_seconds: int | None = None  # Periodic re-analysis (None = disabled)

    # Observation thresholds
    min_trace_occurrences: int = 3            # Min pattern frequency before proposing
    min_determinism_score: float = 0.8        # Min confidence for auto-proposal

    # LLM compiler
    enable_llm_proposals: bool = False        # LLM-assisted proposals (opt-in)
    enable_description_optimization: bool = False  # Description rewrites (opt-in)

    # Governance
    auto_approve_deterministic: bool = False  # Auto-approve fully deterministic chains (dangerous, off by default)
    max_pending_proposals: int = 50           # Cap on pending proposals queue

3. Service pipeline stages

┌─────────────────────────────────────────────────────────────────────┐
│                      ChainWeaverService                            │
│                                                                     │
│  ┌──────────┐    ┌───────────┐    ┌──────────┐    ┌─────────────┐  │
│  │  Trigger  │───►│  Analyze  │───►│ Propose  │───►│  Governance │  │
│  │          │    │           │    │          │    │   Queue      │  │
│  │ • tool   │    │ • schema  │    │ • static │    │             │  │
│  │   change │    │   compat  │    │   chains │    │ • approve   │  │
│  │ • timer  │    │ • pattern │    │ • LLM    │    │ • reject    │  │
│  │ • manual │    │   detect  │    │   flows  │    │ • promote   │  │
│  └──────────┘    └───────────┘    └──────────┘    └──────┬──────┘  │
│                                                          │         │
│                                        ┌─────────────────▼───────┐ │
│                                        │  Expose as Tools / MCP  │ │
│                                        │  (VirtualTool / Server) │ │
│                                        └─────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘

4. Event-driven trigger system

class ServiceEvent(str, Enum):
    TOOL_REGISTERED = "tool_registered"
    TOOL_UPDATED = "tool_updated"           # Schema changed
    TRACE_RECORDED = "trace_recorded"
    ANALYSIS_COMPLETED = "analysis_completed"
    PROPOSAL_CREATED = "proposal_created"
    FLOW_PROMOTED = "flow_promoted"

class ChainWeaverService:
    def on_event(self, event: ServiceEvent, callback: Callable) -> None:
        """Register a callback for service events."""

    def emit(self, event: ServiceEvent, data: dict) -> None:
        """Emit an event to all registered callbacks."""

5. Integration points

Component	Source Issue	Role in Service
ChainAnalyzer	#77	Static schema analysis on trigger
ChainObserver	#78	Runtime trace collection and pattern detection
GovernanceManager	#13	Approval gateway for all proposals
llm_propose_flows()	#28	Optional LLM-assisted flow compilation
optimize_tool_descriptions()	#100	Optional description optimization
VirtualTool.from_flow()	#24	Expose promoted flows as tools
FlowServer	#72	Expose promoted flows as MCP tools

6. CLI integration

Add chainweaver service subcommands (extends #44):

chainweaver service start          # Start the service
chainweaver service status         # Show pending proposals, active flows, metrics
chainweaver service proposals      # List pending proposals with scores
chainweaver service approve <id>   # Approve a proposal
chainweaver service reject <id>    # Reject a proposal
chainweaver service metrics        # Show stats: flows promoted, LLM calls avoided, patterns detected

7. Metrics and reporting

@dataclass
class ServiceMetrics:
    tools_monitored: int
    traces_recorded: int
    patterns_detected: int
    flows_proposed: int
    flows_promoted: int
    total_llm_calls_avoided: int          # Across all executions of promoted flows
    estimated_cost_saved_usd: float
    estimated_latency_saved_ms: float

Relevant Code Locations

• ChainAnalyzer → chainweaver/analyzer.py (Offline computation of valid tool combinations from schemas #77)
• ChainObserver → chainweaver/observer.py (Implement runtime chain observer with auto-flow suggestion #78)
• GovernanceManager → chainweaver/governance.py (Design opt-in governance workflow for promoting observed chains #13)
• llm_propose_flows → chainweaver/compiler_llm.py (Add optional offline LLM-assisted flow compiler (build-time only) #28)
• optimize_tool_descriptions → chainweaver/optimizer.py (Add offline LLM-assisted tool description optimizer #100)
• VirtualTool → chainweaver/virtual_tool.py (Implement Flow-to-VirtualTool adapter for tool-space reduction #24)
• FlowServer → chainweaver/mcp/server.py (MCP Flow Server — expose compiled flows as MCP tools #72)
• CLI → chainweaver/cli.py (Implement CLI entry point with inspect command #44)

Acceptance Criteria

• ChainWeaverService class exists in chainweaver/service.py
• ServiceConfig controls triggers, thresholds, and feature flags
• Tool registration changes trigger re-analysis when configured
• Traces flow through observer → analyzer → proposal pipeline
• All proposals route through GovernanceManager (never auto-registered without governance)
• Event system allows external callbacks on service events
• ServiceMetrics tracks key value-prop numbers (patterns detected, flows promoted, LLM calls avoided)
• Service can run as a background thread or standalone process
• LLM-assisted features (compiler, description optimizer) are opt-in via config
• auto_approve_deterministic is False by default (safety)
• At least 8 test cases: tool change triggers analysis, pattern detection triggers proposal, proposal routes to governance, promote creates virtual tool, metrics tracking, event callbacks, config toggles, manual trigger
• No new required dependencies (uses existing ChainWeaver components)

Out of Scope

• Web UI / dashboard (can be built on top of the service API)
• Distributed / multi-node service
• Persistent state across restarts (uses in-memory components; persistence comes from Implement persistent registry adapters (protocol + file-based store) #16 FileStore)
• Authentication / multi-tenant isolation
• Rate limiting of LLM calls (user's responsibility via llm_fn)

Dependencies

This is an integration issue that ties together many prior components:

Dependency	Issue	Milestone	Status
ChainAnalyzer	#77	M3 (v0.3.0)	Open
ChainObserver	#78	M4 (v0.4.0)	Open
GovernanceManager	#13	M3 (v0.3.0)	Open
VirtualTool	#24	M6 (v0.6.0)	Open
compiler_llm.py	#28	M7 (v0.7.0)	Open
optimizer.py	#100	M7 (v0.7.0)	Open
CLI entry point	#44	M7 (v0.7.0)	Open

Minimum viable service: Requires only #77 (analyzer) + #13 (governance). LLM compiler, observer, and description optimizer can be plugged in incrementally.

Notes

• This is the product-level manifestation of ChainWeaver's core value proposition: instead of manual flow authoring, the system continuously discovers and proposes optimizations.
• The service transforms ChainWeaver from "a library you call" to "a layer that improves your agents automatically."
• The estimated_llm_calls_avoided metric in ServiceMetrics is the single most compelling adoption argument — it directly quantifies value.
• Consider a "dry-run mode" where the service runs analysis but only logs proposals without creating governance entries — useful for evaluation before commitment.
• The event system enables future integrations: Slack notifications on proposals, CI hooks on promotions, dashboard updates on metrics changes.