Context
The way matching system uses additive triggering — multiple ways can fire on a single prompt when they share relevant vocabulary. This is by design: a prompt like "migrate the database schema" legitimately needs guidance from both delivery/migrations (schema change mechanics) and architecture/design (target architecture).
Currently the test framework (fixture + integration) measures single-expected-way accuracy. It validates that the right way fires, but doesn't measure whether the set of co-activated ways is appropriate.
What's needed
A test layer that validates co-activation quality:
- Prompts designed to trigger 2+ related ways, with expected sets
- Measurement of whether co-activated ways provide complementary (not redundant) context
- Detection of unwanted co-activation (unrelated ways bleeding in)
The activation test (step 5) now tests co-activation live, but the automated harness doesn't yet support multi-expected-way fixtures.
Suggested approach
Extend test-fixtures.jsonl format to support "expected": ["way-a", "way-b"] arrays alongside the existing single-string format. Add co-activation test cases for known overlap areas: schema/design, error/debugging, security/deps.
Context
The way matching system uses additive triggering — multiple ways can fire on a single prompt when they share relevant vocabulary. This is by design: a prompt like "migrate the database schema" legitimately needs guidance from both
delivery/migrations(schema change mechanics) andarchitecture/design(target architecture).Currently the test framework (fixture + integration) measures single-expected-way accuracy. It validates that the right way fires, but doesn't measure whether the set of co-activated ways is appropriate.
What's needed
A test layer that validates co-activation quality:
The activation test (step 5) now tests co-activation live, but the automated harness doesn't yet support multi-expected-way fixtures.
Suggested approach
Extend
test-fixtures.jsonlformat to support"expected": ["way-a", "way-b"]arrays alongside the existing single-string format. Add co-activation test cases for known overlap areas: schema/design, error/debugging, security/deps.