Comprehensive dedup quality: type/project/source awareness#268
Merged
CalebisGross merged 1 commit intomainfrom Mar 20, 2026
Merged
Comprehensive dedup quality: type/project/source awareness#268CalebisGross merged 1 commit intomainfrom
CalebisGross merged 1 commit intomainfrom
Conversation
Refactor the encoding agent's dedup logic to prevent false merges: 1. Raise default threshold 0.9 → 0.95 — reduces false positives by 46% based on lifecycle test data (355 of 765 deduped memories were below 0.95 similarity) 2. Type-aware: never dedup across memory types — a decision and an error about the same topic are distinct memories worth keeping 3. Project-aware: never dedup across projects — "chose SQLite" in mnemonic and "chose SQLite" in felix-lm are different decisions 4. Source-aware: MCP memories (explicit user/agent input) use a higher threshold of 0.98 — deliberate input deserves preservation 5. Clearer dedup feedback in check_memory and remember responses The findDuplicate function now takes a dedupContext struct instead of a bare threshold, making the dedup rules explicit and testable. 10 unit tests covering all dedup scenarios (up from 4). Closes #266 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Systematic fix for false memory deduplication. The encoding agent was merging distinct memories that shared vocabulary (e.g., two different decisions about SQLite, or a decision vs an error about the same topic).
Changes
Evidence
From the real-LLM lifecycle baseline (#262):
Architecture
Replaced bare
float32threshold withdedupContextstruct:Both encoding paths (polling + event-driven) use
buildDedupContext()→findDuplicate().Test plan
make build+make testpassgolangci-lint run— 0 issuesCloses #266