Skip to content

fix(relay): make filters with an empty set match nothing in SQL - #9233

Draft
ThePumpingLemma wants to merge 1 commit into
mainfrom
dgrochowski/empty-filter-sets-match-nothing
Draft

ThePumpingLemma wants to merge 1 commit into
mainfrom
dgrochowski/empty-filter-sets-match-nothing

Conversation

@ThePumpingLemma

@ThePumpingLemma ThePumpingLemma commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Problem

NIP-01 matching (buzz_core::filter::filters_match) treats a present-but-empty kinds, authors, ids or tag set as matching nothing. It does the same for a since later than any time the store can represent. filter_to_query_params builds the SQL query for WS REQ/COUNT and HTTP /query//count, and it agreed only for kinds: [], which it keeps as Some([]), the store's match-nothing query. Every other case became "no constraint":

  • empty authors, ids and #e became None
  • empty #p, #t, #h, #d and other tag sets were dropped
  • an unrepresentable since dropped the lower bound

filter_fully_pushable accepts those filters: empty #p and empty generic tags pass, and authors/ids are never checked. So COUNT takes the SQL fast path, and {"kinds":[9],"authors":[]} or {"kinds":[9],"#t":[]} counts every kind-9 event in the reader's scope instead of 0. This happens on both WS COUNT and HTTP /count. REQ scans a full page, and the in-memory match then throws all of it away.

since/until were also cast with as i64, so u64::MAX wrapped to -1:

  • such a since matched everything
  • such an until set a bound before the epoch, so REQ returned nothing where NIP-01 matches everything

The NIP-50 search loops (WS and HTTP) turned kinds: [] and authors: [] into "no constraint" too. They then ran full-text pages that the match discarded.

Change

  • filter_matches_nothing(filter) is now the single check for whether a filter can match no event. It mirrors filters_match.
  • filter_timestamp(ts) converts a filter timestamp to a store timestamp. It returns None when the store cannot represent the value, and it never wraps.
  • filter_to_query_params turns a filter that matches nothing into the existing match-nothing query, kinds: Some([]). Every store query and count entry point already answers that query without running SQL. since/until now go through filter_timestamp.
  • The WS and HTTP NIP-50 search loops skip filters that match nothing. That left their empty-set handling dead, so they now map kinds/authors directly, scope by #h without an empty-set guard, and convert since/until through filter_timestamp.

Filters with non-empty sets build the same query as before. An unrepresentable until is still no upper bound. Live fan-out is unchanged, because it matches in memory. No DB or schema change.

Testing

New test: handlers::req::tests::filter_that_matches_nothing_builds_a_match_nothing_query. COUNT and /count build their query through build_event_query_from_filter, which calls filter_to_query_params; the test calls filter_to_query_params directly. For each empty-set case, plus since of 9e12 and of u64::MAX, it checks two things:

  • the in-memory match rejects a kind-9 event that the {"kinds":[9]} control matches
  • the built query is the match-nothing query

It also checks that until: u64::MAX leaves the query unbounded.

Before the fix:

$ cargo test -p buzz-relay --lib filter_that_matches_nothing_builds_a_match_nothing_query
test handlers::req::tests::filter_that_matches_nothing_builds_a_match_nothing_query ... FAILED
assertion `left == right` failed: {"authors":[],"kinds":[9]} must build the match-nothing query
  left: Some([9])
 right: Some([])
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2396 filtered out

After the fix:

$ cargo fmt --all                                            # clean
$ cargo clippy -p buzz-relay --all-targets -- -D warnings    # Finished, no warnings
$ cargo test -p buzz-relay --lib -- handlers:: api::bridge
test result: ok. 548 passed; 0 failed; 278 ignored; 0 measured; 1571 filtered out

After rebasing onto the current main, the pre-push hook's just test-unit passed on this commit. A first push attempt failed one step, membership_cache::tests::, while other builds were loading the machine. That module passes on its own (11 passed) and this change does not touch it.

Not covered

  • I have not yet run end-to-end COUNT/REQ checks against a local relay. The Postgres-backed (ignored) tests were not run either.
  • The HTTP bridge feed (feed_types extension) still treats an unrepresentable since as no lower bound. The feed does not use NIP-01 matching: it ignores kinds, authors and tags. Moving it to filter_timestamp would change nothing you can observe, and returning an empty feed for such a since would change the feed contract. Both are out of scope here.
  • The fast COUNT path still checks out a pool connection before count_events_on returns 0 for the match-nothing query. Only the non-count entry points answer it before the acquire.

Review

  • filter_matches_nothing was checked field by field against filters_match, including the #h channel-id fallback and the membership-notice #t rule. Every REQ, COUNT, /query and /count path reaches it through filter_to_query_params, and the two NIP-50 search loops skip the filter up front. Blockers: none.
  • Fixed in review: both search loops still guarded #h with .filter(|vs| !vs.is_empty()), and the match-nothing skip made that guard dead. It has been removed. Re-ran cargo fmt --all, cargo clippy -p buzz-relay --all-targets -- -D warnings, and cargo test -p buzz-relay --lib -- handlers:: api::bridge (548 passed). Amended as ae7d4ec0b.
  • Open, optional:
    • Add a non-empty control for each field in the table test, so that a predicate that over-matches fails it.
    • Drop the per-field empty guards in filter_to_query_params. They are now moot.

Exercised on a local relay

Commit tested: ae7d4ec0b. Dev profile, isolated harness relay on :3030, authenticated channel member. The channel held 6 kind-9 events; 5 of them carried #t=tag. Filters were sent as raw JSON so u64::MAX arrived exactly.

Filter WS COUNT HTTP /count WS REQ HTTP /query
{kinds:[9],authors:[]} 0 0 EOSE, 0 events 0
{kinds:[9],"#t":[]} 0 0 EOSE, 0 0
{kinds:[9],ids:[]} 0 0 EOSE, 0 0
{kinds:[9],"#p":[]} 0 0 EOSE, 0 0
{kinds:[9],since:u64::MAX} 0 0 EOSE, 0 0
{kinds:[9],authors:[],search:"m1"} n/a n/a EOSE, 0 0
control {kinds:[9]} 6 6 EOSE, 6 6
control {kinds:[9],authors:[me]} 6 6 EOSE, 6 6
control {kinds:[9],"#t":["tag"]} 5 5 EOSE, 5 5
{kinds:[9],until:u64::MAX} 6 6 EOSE, 6 6

Filters with an empty set, or with a since the store can't represent, count 0 and return nothing on both transports. The controls return the real counts. An unrepresentable until is still treated as no upper bound.

Postgres-gated tests: the PR adds or modifies none. As a related check I ran cargo test -p buzz-relay --lib -- --ignored --test-threads=1 count search, pointing DATABASE_URL, BUZZ_TEST_DATABASE_URL and BUZZ_TEST_REDIS_URL at the harness: 21 passed, 0 failed.

Not exercised:

  • A before/after run against main.
  • Empty #e/#d tag sets; the unit test covers them.

Problem: NIP-01 matching (`buzz_core::filter::filters_match`) rejects every
event for a filter with a present-but-empty `kinds`, `authors`, `ids` or tag
set, and for a `since` later than any time the store can represent.
`filter_to_query_params`, which builds the SQL query for WS REQ/COUNT and
HTTP /query and /count, agreed only for `kinds: []` (kept as `Some([])`, the
store's match-nothing query). Empty `authors`, `ids` and `#e` became `None`,
empty `#p`, `#t`, `#h`, `#d` and other tag sets were dropped, and an
unrepresentable `since` dropped the lower bound, so each turned into "no
constraint".

How it fails: `filter_fully_pushable` accepts those filters (empty `#p` and
empty generic tags pass, `authors`/`ids` are never checked), so COUNT takes
the SQL fast path and `{"kinds":[9],"authors":[]}` or `{"kinds":[9],"#t":[]}`
counts every kind-9 event in the reader's scope instead of 0, on WS COUNT and
HTTP /count. REQ scans a full page that the in-memory match then discards.
`since` and `until` were also cast with `as i64`, so `u64::MAX` wrapped to
-1: such a `since` matched everything, and such an `until` bounded the query
before the epoch, so REQ returned nothing where NIP-01 matches everything.
The NIP-50 search paths (WS and HTTP) turned `kinds: []` and `authors: []`
into "no constraint" and ran full-text pages that the match discarded.

Change: one predicate, `filter_matches_nothing`, states when a filter can
match no event, and one conversion, `filter_timestamp`, maps a filter
timestamp to a store timestamp (`None` when unrepresentable, without
wrapping). `filter_to_query_params` turns a filter that matches nothing into
the existing match-nothing query (`kinds: Some([])`), which every store
query and count entry point already answers without SQL, and converts
`since`/`until` through `filter_timestamp`. The WS and HTTP search loops skip
such filters, which makes their empty-set handling dead, so they now map
`kinds`/`authors` directly, scope by `#h` without an empty-set guard, and
convert `since`/`until` the same way.

Why safe: the predicate mirrors `filters_match`, so every filter it flags
was already returning no events through the post-filter (or overcounting
through the fast COUNT path); the new test asserts both sides agree. Filters
with non-empty sets build the same query as before. An unrepresentable
`until` is still no upper bound. Live fan-out is unchanged; it matches in
memory.

Tests: `filter_that_matches_nothing_builds_a_match_nothing_query` builds
the query through `filter_to_query_params` (the function behind
`build_event_query_from_filter`, which COUNT and HTTP /count use) for each
empty-set case and both unrepresentable `since` values, asserts the
in-memory match rejects a kind-9 event that the `{"kinds":[9]}` control
matches, and asserts the query matches nothing; it also pins that
`until: u64::MAX` leaves the query unbounded. Before the fix it failed on
`{"authors":[],"kinds":[9]}` (query kinds `Some([9])`, not `Some([])`).

Signed-off-by: David Grochowski <dgrochowski@squareup.com>
@ThePumpingLemma
ThePumpingLemma force-pushed the dgrochowski/empty-filter-sets-match-nothing branch from c4a2ff1 to ae7d4ec Compare November 7, 2026 04:59

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant