Skip to content

Conversation

@dluc
Copy link
Collaborator

@dluc dluc commented Dec 2, 2025

Summary

Fixes the NOT operator issues - the NOT operator now works correctly using a hybrid FTS + LINQ approach.

Problem

The NOT operator had two problems:

  1. Standalone NOT crashed: km search "NOT foo" threw FTS5 syntax error
  2. NOT didn't exclude: km search "foo AND NOT bar" returned documents containing both

Solution

Hybrid FTS + LINQ filtering:

  • FTS5 handles positive terms only (no NOT terms sent to FTS)
  • NOT terms are collected separately and excluded via LINQ post-filtering
  • Standalone NOT queries work by fetching all documents then filtering

Changes

  • src/Core/Search/NodeSearchService.cs:

    • Added FtsQueryResult record to separate FTS query from NOT terms
    • Modified FtsQueryExtractor to collect NOT terms separately
    • Added ApplyNotTermFiltering() for LINQ exclusion
  • src/Core/Search/SqliteFtsIndex.cs:

    • Added GetAllDocumentsAsync() for standalone NOT queries
  • src/Core/Search/Query/Parsers/InfixQueryParser.cs:

    • Added single-quote support (was treating 'AND' as literal with quotes)
  • src/Main/CLI/Commands/ExamplesCommand.cs:

    • Added NOT operator examples (infix and JSON format)
  • tests/Core.Tests/Search/SearchEndToEndTests.cs:

    • Added 9 E2E tests for NOT operator scenarios
  • KNOWN-ISSUES.md: Cleaned up, only open issues remain

Usage

# Standalone NOT - find documents NOT containing "secret"
km search "NOT secret"

# Exclusion - find "foo" but NOT "bar"
km search "foo AND NOT bar"

# JSON format
km search '{"content": {"$not": "deprecated"}}'
km search '{"$nor": [{"content": "alpha"}, {"content": "beta"}]}'

Note: Use explicit AND between terms (e.g., foo AND NOT bar)

Test plan

  • All 546 tests pass (332 Core + 214 Main)
  • Zero skipped tests
  • build.sh passes with 0 warnings
  • format.sh passes

dluc added 4 commits December 2, 2025 13:41
The NOT operator now works correctly using a hybrid FTS + LINQ approach:
- FTS5 handles positive terms only
- NOT terms are excluded via LINQ post-filtering
- Standalone NOT queries now work (previously crashed)

Changes:
- Add FtsQueryResult record to separate FTS query from NOT terms
- Modify FtsQueryExtractor to collect NOT terms separately
- Add ApplyNotTermFiltering for LINQ exclusion
- Add GetAllDocumentsAsync for standalone NOT queries
- Add 7 E2E tests for NOT operator scenarios
- Mark known issue as resolved in KNOWN-ISSUES.md

Usage note: Use explicit AND between terms, e.g., "foo AND NOT bar"
The tokenizer now handles both double-quoted ("...") and single-quoted
('...') strings. Previously, single quotes were treated as part of
identifiers, causing NOT 'AND' to search for literal "'AND'" instead
of "AND".
Keep only open issues. Resolved issues don't need to be tracked here.
Show how to exclude terms with NOT and standalone NOT queries.
@dluc dluc force-pushed the fix-not-operator branch from 4741a42 to fdcdf7d Compare December 2, 2025 13:17
@dluc dluc merged commit 5836215 into main Dec 2, 2025
7 checks passed
@dluc dluc deleted the fix-not-operator branch December 2, 2025 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants