feat(weave): WAL tee middleware for durable API writes#6387
feat(weave): WAL tee middleware for durable API writes#6387andrewtruong wants to merge 37 commits intomasterfrom
Conversation
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… 'origin/andrew/durable-wal-client' into andrew/wal-tee-writes
Introduces WALTeeTraceServer, a middleware that intercepts all write operations (calls, objects, tables, files, feedback, costs) and appends them to a local JSONL WAL before delegating to the underlying trace server. This is the "tee" layer that makes writes durable — if the process crashes, the WAL can be replayed to recover unacknowledged data. - New `WALTeeTraceServer` using `DelegatingTraceServerMixin` pattern - New `enable_wal` / `WEAVE_ENABLE_WAL` setting (default: off) - Wired into `weave_init.py` as outermost middleware - WAL writes are fire-and-forget (never block the real request) - 30 unit tests covering tee, delegation, error handling, serialization Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
|
Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=62746059f8aab19d45fa1df0979213d49073e9a0 |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| from pathlib import Path | ||
|
|
||
| from weave.durability.wal_directory_manager import FileWALDirectoryManager | ||
| from weave.durability.wal_tee_trace_server import WALTeeTraceServer | ||
| from weave.durability.wal_writer import JSONLWALWriter |
There was a problem hiding this comment.
🔴 Imports inside function body violate mandatory AGENTS.md import rule
The _wrap_with_wal function places four imports inside the function body (from pathlib import Path, and three weave.durability imports). AGENTS.md explicitly requires all imports at the module level and states the only exceptions are circular import avoidance (documented with a comment), optional dependencies (wrapped in try/except), and TYPE_CHECKING. None of these exceptions apply here: pathlib is stdlib, and the weave.durability modules are part of the same package — not optional dependencies that may not be installed. The AGENTS.md specifically warns that "Imports inside functions are not caught by linting. Agents must self-enforce this rule."
Prompt for agents
Move the four imports inside _wrap_with_wal (lines 247-251 of weave/trace/weave_init.py) to the top of the file, alongside the other top-level imports. Specifically, move:
from pathlib import Path
from weave.durability.wal_directory_manager import FileWALDirectoryManager
from weave.durability.wal_tee_trace_server import WALTeeTraceServer
from weave.durability.wal_writer import JSONLWALWriter
to the import section at the top of weave/trace/weave_init.py (around lines 1-27). The function body should then reference them directly without any local imports.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
WALTeeTraceServermiddleware that tees all write operations to a local JSONL WAL file before delegating to the trace serverenable_wal/WEAVE_ENABLE_WALsetting (default off) to opt inweave_init.pyas the outermost middleware layer, so the WAL captures all write intentsDepends on:
andrew/wire-digest-client) — client-side digest computationandrew/durable-wal-client— JSONL WAL writer/consumer/directory managerHow it works
The tee sits in the middleware chain:
WALTee → Cache → RemoteHTTPTraceServer. For every intercepted write method (call_start/end, obj_create, table_create, file_create, feedback_create, cost_create, etc.), it:{"type": "<method_name>", "req": {...}}to the WALWAL writes are fire-and-forget — if WAL I/O fails, the error is logged and the request proceeds normally. This ensures the WAL never blocks normal operation.
Test plan
🤖 Generated with Claude Code