Repository navigation
fix(relay): make filters with an empty set match nothing in SQL - #9233
Draft
ThePumpingLemma wants to merge 1 commit into
Draft
ThePumpingLemma wants to merge 1 commit into
ThePumpingLemma wants to merge 1 commit into
Conversation
Problem: NIP-01 matching (`buzz_core::filter::filters_match`) rejects every
event for a filter with a present-but-empty `kinds`, `authors`, `ids` or tag
set, and for a `since` later than any time the store can represent.
`filter_to_query_params`, which builds the SQL query for WS REQ/COUNT and
HTTP /query and /count, agreed only for `kinds: []` (kept as `Some([])`, the
store's match-nothing query). Empty `authors`, `ids` and `#e` became `None`,
empty `#p`, `#t`, `#h`, `#d` and other tag sets were dropped, and an
unrepresentable `since` dropped the lower bound, so each turned into "no
constraint".
How it fails: `filter_fully_pushable` accepts those filters (empty `#p` and
empty generic tags pass, `authors`/`ids` are never checked), so COUNT takes
the SQL fast path and `{"kinds":[9],"authors":[]}` or `{"kinds":[9],"#t":[]}`
counts every kind-9 event in the reader's scope instead of 0, on WS COUNT and
HTTP /count. REQ scans a full page that the in-memory match then discards.
`since` and `until` were also cast with `as i64`, so `u64::MAX` wrapped to
-1: such a `since` matched everything, and such an `until` bounded the query
before the epoch, so REQ returned nothing where NIP-01 matches everything.
The NIP-50 search paths (WS and HTTP) turned `kinds: []` and `authors: []`
into "no constraint" and ran full-text pages that the match discarded.
Change: one predicate, `filter_matches_nothing`, states when a filter can
match no event, and one conversion, `filter_timestamp`, maps a filter
timestamp to a store timestamp (`None` when unrepresentable, without
wrapping). `filter_to_query_params` turns a filter that matches nothing into
the existing match-nothing query (`kinds: Some([])`), which every store
query and count entry point already answers without SQL, and converts
`since`/`until` through `filter_timestamp`. The WS and HTTP search loops skip
such filters, which makes their empty-set handling dead, so they now map
`kinds`/`authors` directly, scope by `#h` without an empty-set guard, and
convert `since`/`until` the same way.
Why safe: the predicate mirrors `filters_match`, so every filter it flags
was already returning no events through the post-filter (or overcounting
through the fast COUNT path); the new test asserts both sides agree. Filters
with non-empty sets build the same query as before. An unrepresentable
`until` is still no upper bound. Live fan-out is unchanged; it matches in
memory.
Tests: `filter_that_matches_nothing_builds_a_match_nothing_query` builds
the query through `filter_to_query_params` (the function behind
`build_event_query_from_filter`, which COUNT and HTTP /count use) for each
empty-set case and both unrepresentable `since` values, asserts the
in-memory match rejects a kind-9 event that the `{"kinds":[9]}` control
matches, and asserts the query matches nothing; it also pins that
`until: u64::MAX` leaves the query unbounded. Before the fix it failed on
`{"authors":[],"kinds":[9]}` (query kinds `Some([9])`, not `Some([])`).
Signed-off-by: David Grochowski <dgrochowski@squareup.com>
ThePumpingLemma
force-pushed
the
dgrochowski/empty-filter-sets-match-nothing
branch
from
November 7, 2026 04:59
c4a2ff1 to
ae7d4ec
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
NIP-01 matching (
buzz_core::filter::filters_match) treats a present-but-emptykinds,authors,idsor tag set as matching nothing. It does the same for asincelater than any time the store can represent.filter_to_query_paramsbuilds the SQL query for WSREQ/COUNTand HTTP/query//count, and it agreed only forkinds: [], which it keeps asSome([]), the store's match-nothing query. Every other case became "no constraint":authors,idsand#ebecameNone#p,#t,#h,#dand other tag sets were droppedsincedropped the lower boundfilter_fully_pushableaccepts those filters: empty#pand empty generic tags pass, andauthors/idsare never checked. SoCOUNTtakes the SQL fast path, and{"kinds":[9],"authors":[]}or{"kinds":[9],"#t":[]}counts every kind-9 event in the reader's scope instead of 0. This happens on both WSCOUNTand HTTP/count.REQscans a full page, and the in-memory match then throws all of it away.since/untilwere also cast withas i64, sou64::MAXwrapped to -1:sincematched everythinguntilset a bound before the epoch, soREQreturned nothing where NIP-01 matches everythingThe NIP-50 search loops (WS and HTTP) turned
kinds: []andauthors: []into "no constraint" too. They then ran full-text pages that the match discarded.Change
filter_matches_nothing(filter)is now the single check for whether a filter can match no event. It mirrorsfilters_match.filter_timestamp(ts)converts a filter timestamp to a store timestamp. It returnsNonewhen the store cannot represent the value, and it never wraps.filter_to_query_paramsturns a filter that matches nothing into the existing match-nothing query,kinds: Some([]). Every store query and count entry point already answers that query without running SQL.since/untilnow go throughfilter_timestamp.kinds/authorsdirectly, scope by#hwithout an empty-set guard, and convertsince/untilthroughfilter_timestamp.Filters with non-empty sets build the same query as before. An unrepresentable
untilis still no upper bound. Live fan-out is unchanged, because it matches in memory. No DB or schema change.Testing
New test:
handlers::req::tests::filter_that_matches_nothing_builds_a_match_nothing_query.COUNTand/countbuild their query throughbuild_event_query_from_filter, which callsfilter_to_query_params; the test callsfilter_to_query_paramsdirectly. For each empty-set case, plussinceof 9e12 and ofu64::MAX, it checks two things:{"kinds":[9]}control matchesIt also checks that
until: u64::MAXleaves the query unbounded.Before the fix:
After the fix:
After rebasing onto the current
main, the pre-push hook'sjust test-unitpassed on this commit. A first push attempt failed one step,membership_cache::tests::, while other builds were loading the machine. That module passes on its own (11 passed) and this change does not touch it.Not covered
COUNT/REQchecks against a local relay. The Postgres-backed (ignored) tests were not run either.feed_typesextension) still treats an unrepresentablesinceas no lower bound. The feed does not use NIP-01 matching: it ignoreskinds,authorsand tags. Moving it tofilter_timestampwould change nothing you can observe, and returning an empty feed for such asincewould change the feed contract. Both are out of scope here.COUNTpath still checks out a pool connection beforecount_events_onreturns 0 for the match-nothing query. Only the non-count entry points answer it before the acquire.Review
filter_matches_nothingwas checked field by field againstfilters_match, including the#hchannel-id fallback and the membership-notice#trule. Every REQ, COUNT,/queryand/countpath reaches it throughfilter_to_query_params, and the two NIP-50 search loops skip the filter up front. Blockers: none.#hwith.filter(|vs| !vs.is_empty()), and the match-nothing skip made that guard dead. It has been removed. Re-rancargo fmt --all,cargo clippy -p buzz-relay --all-targets -- -D warnings, andcargo test -p buzz-relay --lib -- handlers:: api::bridge(548 passed). Amended asae7d4ec0b.filter_to_query_params. They are now moot.Exercised on a local relay
Commit tested:
ae7d4ec0b. Dev profile, isolated harness relay on :3030, authenticated channel member. The channel held 6 kind-9 events; 5 of them carried#t=tag. Filters were sent as raw JSON sou64::MAXarrived exactly.COUNT/countREQ/query{kinds:[9],authors:[]}{kinds:[9],"#t":[]}{kinds:[9],ids:[]}{kinds:[9],"#p":[]}{kinds:[9],since:u64::MAX}{kinds:[9],authors:[],search:"m1"}{kinds:[9]}{kinds:[9],authors:[me]}{kinds:[9],"#t":["tag"]}{kinds:[9],until:u64::MAX}Filters with an empty set, or with a
sincethe store can't represent, count 0 and return nothing on both transports. The controls return the real counts. An unrepresentableuntilis still treated as no upper bound.Postgres-gated tests: the PR adds or modifies none. As a related check I ran
cargo test -p buzz-relay --lib -- --ignored --test-threads=1 count search, pointingDATABASE_URL,BUZZ_TEST_DATABASE_URLandBUZZ_TEST_REDIS_URLat the harness: 21 passed, 0 failed.Not exercised:
#e/#dtag sets; the unit test covers them.