Problem Statement
The filter indexes encode compound keys with : as a delimiter. If user-provided
field names or category values also contain :, distinct logical (field, value) pairs can collide at the byte-key level. That can produce incorrect
filter matches.
Users are also affected by inconsistent validation behavior: malformed JSON,
unsupported field values, schema type mismatches, invalid operators, and invalid
range shapes should be rejected clearly rather than being skipped or producing
undefined filter state.
Proposed Solution
Reject unsafe or invalid filter inputs before reading or writing the filter
indexes:
- malformed filter JSON;
- unsupported filter value types;
- empty field names;
- schema type mismatches;
- invalid operators;
- invalid numeric range shape or order;
: in filter field names;
: in category values.
These rejections should be caller-fixable validation errors and should propagate
to API callers as HTTP 400 responses.
Alternatives Considered
Switch immediately to a length-prefixed or binary tuple key format. That is more
robust long term, but it is a breaking on-disk format change and needs migration
or rebuild tooling.
Escape : inside user fields and values. That avoids rejecting input, but still
requires a careful compatibility story for existing keys and makes key handling
more complex.
Problem Statement
The filter indexes encode compound keys with
:as a delimiter. If user-providedfield names or category values also contain
:, distinct logical(field, value)pairs can collide at the byte-key level. That can produce incorrectfilter matches.
Users are also affected by inconsistent validation behavior: malformed JSON,
unsupported field values, schema type mismatches, invalid operators, and invalid
range shapes should be rejected clearly rather than being skipped or producing
undefined filter state.
Proposed Solution
Reject unsafe or invalid filter inputs before reading or writing the filter
indexes:
:in filter field names;:in category values.These rejections should be caller-fixable validation errors and should propagate
to API callers as HTTP 400 responses.
Alternatives Considered
Switch immediately to a length-prefixed or binary tuple key format. That is more
robust long term, but it is a breaking on-disk format change and needs migration
or rebuild tooling.
Escape
:inside user fields and values. That avoids rejecting input, but stillrequires a careful compatibility story for existing keys and makes key handling
more complex.