Skip to content

Tune vocabulary for newly-semantic ways via real session data #29

@aaronsb

Description

@aaronsb

Context

11 ways gained BM25 semantic matching in the taxonomy restructure. Their initial vocabularies were authored based on way content and likely user prompts, but haven't been tested against real session data.

What's needed

Run `/test-way suggest --all` after a few sessions to see:

  • Gaps: body terms that users actually say but aren't in vocabulary
  • Unused terms: vocabulary terms that never appear in prompts (may be fine — they catch user language, not body language)
  • Cross-talk: terms shared between ways that cause unwanted co-activation

Priority ways to tune (most likely to have cross-talk):

  • delivery/migrations vs architecture/design (shared: schema, database)
  • code/errors vs environment/debugging (shared: error, exception)
  • code/security vs environment/deps (shared: vulnerability, audit)
  • delivery/commits vs delivery/release (shared: version-adjacent terms)

Approach

Use `/test-way score-all` with ambiguous prompts to check discrimination. Threshold adjustments (raising from default 2.0) may be needed for ways with broader vocabularies. The hard constraint: 0 FP in the test harness.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions