From 2a7a490567ec3a81a86fd34d430ddd948535de73 Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 7 May 2026 10:15:51 +1000 Subject: [PATCH 1/7] fix(ope): address CodeRabbit and reviewer feedback on PR #176 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Drop dead NULL-handling branches from compare_ope_cllw_* (STRICT short-circuits NULL inputs before the body runs). - Fix ciphertext-length wording in OPE type docs ("8 bytes per plaintext byte", not "per plaintext bit", which contradicted the 65-byte result). - Convert simple OPE helpers (extractors, has_*, order_by_ope) from LANGUAGE plpgsql to LANGUAGE sql so the planner can inline them on the sort/order_by hot path. Per project guideline. - Spell the cipher consistently as CLLW (matching ope_cllw_* type names and the existing ore_cllw_* docs); CLWW was a typo. - Clarify sql-support.md MIN/MAX and COUNT(DISTINCT) rows so ope is no longer implied to be a ste_vec-extracted node term — it only applies to the outer column. - Loosen brittle exact-list assertion in config_check_rejects_unknown_index to mention 'ope' and the offending token. - Add ore_wins_over_opf_when_both_present to lock in the ORE-before-OPE dispatch precedence in eql_v2.compare. - Add operator-level coverage for <=, >=, =, <> on both opf and opv variants (previously only < and > were covered for opf, and opv was exercised through compare_* but not via operators). --- docs/reference/index-config.md | 2 +- docs/reference/sql-support.md | 6 +- src/ope_cllw_u64_65/compare.sql | 24 +--- src/ope_cllw_u64_65/functions.sql | 44 +++--- src/ope_cllw_u64_65/types.sql | 8 +- src/ope_cllw_var_8/compare.sql | 24 +--- src/ope_cllw_var_8/functions.sql | 44 +++--- src/ope_cllw_var_8/types.sql | 4 +- src/operators/order_by.sql | 18 +-- tests/sqlx/tests/ope_tests.rs | 224 +++++++++++++++++++++++++++++- 10 files changed, 282 insertions(+), 116 deletions(-) diff --git a/docs/reference/index-config.md b/docs/reference/index-config.md index c5e01746..094f39de 100644 --- a/docs/reference/index-config.md +++ b/docs/reference/index-config.md @@ -114,7 +114,7 @@ Try to ensure that the string you search for is at least as long as the `tokenLe Both `ore` and `ope` enable the same ordered-comparison surface (`<`, `<=`, `=`, `>`, `>=`, `BETWEEN`, `ORDER BY`, `MIN`/`MAX`). - **`ore`** uses Order-Revealing Encryption (`ore_block_u64_8_256`, payload field `ob`). Ciphertexts compare via a custom per-byte protocol implemented in `eql_v2.compare_ore_block_u64_8_256`. This is the default ordered-search index. -- **`ope`** uses CLWW Order-Preserving Encryption — `ope_cllw_u64_65` (fixed-width, payload field `opf`) for numeric types and `ope_cllw_var_8` (variable-width, payload field `opv`) for text-shaped values. OPE ciphertexts compare with **standard lexicographic byte ordering**, which makes them usable in environments that can only sort `bytea` natively (e.g. some pluggable storage layers without custom comparators). +- **`ope`** uses CLLW Order-Preserving Encryption — `ope_cllw_u64_65` (fixed-width, payload field `opf`) for numeric types and `ope_cllw_var_8` (variable-width, payload field `opv`) for text-shaped values. OPE ciphertexts compare with **standard lexicographic byte ordering**, which makes them usable in environments that can only sort `bytea` natively (e.g. some pluggable storage layers without custom comparators). `eql_v2.compare()` and the `<` / `<=` / `>` / `>=` operators dispatch automatically to whichever ordered terms are present on the encrypted value, so application queries do not change when switching between `ore` and `ope`. diff --git a/docs/reference/sql-support.md b/docs/reference/sql-support.md index 5a11e430..fc1f25e0 100644 --- a/docs/reference/sql-support.md +++ b/docs/reference/sql-support.md @@ -12,7 +12,7 @@ EQL ships five search index kinds that encrypt data in ways that preserve specif | `match` | `bloom_filter` (`bf`) | Substring / token matching via `LIKE` / `ILIKE` | | `ste_vec` | Structured encryption (`sv`) | JSONB containment and JSONB path / field access | -> **`ore` vs `ope`** — both index kinds support the same ordered-comparison surface. `ore` (Order-Revealing Encryption) is the default. `ope` (CLWW Order-Preserving Encryption) is an alternative for environments that need plain lexicographic byte comparison (e.g. pluggable storage that cannot run a custom comparator). On a column configured for `ope`, `eql_v2.compare()` and the `<` / `<=` / `>` / `>=` operators dispatch to OPE terms automatically. +> **`ore` vs `ope`** — both index kinds support the same ordered-comparison surface. `ore` (Order-Revealing Encryption) is the default. `ope` (CLLW Order-Preserving Encryption) is an alternative for environments that need plain lexicographic byte comparison (e.g. pluggable storage that cannot run a custom comparator). On a column configured for `ope`, `eql_v2.compare()` and the `<` / `<=` / `>` / `>=` operators dispatch to OPE terms automatically. Every column must also be registered with `eql_v2.add_column(...)` — that alone gives the column storage and decryption, but none of the operators below will produce results until at least one search index is added for the operation you need. @@ -139,8 +139,8 @@ When the `ste_vec` index is configured, CipherStash Proxy rewrites these standar | `jsonb_array_elements(arr)` | `eql_v2.jsonb_array_elements(arr)` | Path must resolve to a JSON array node | Set-returning; yields `eql_v2_encrypted`. | | `jsonb_array_elements_text(arr)` | `eql_v2.jsonb_array_elements_text(arr)` | Path must resolve to a JSON array node | Set-returning; yields ciphertext as `text`. | | `COUNT(col)` | plain `count(*)` | — | No encrypted term required. | -| `COUNT(DISTINCT col)` | deterministic dedup | `unique`, `ore`, **or** `ope` on the extracted node | For a JSON leaf, that means Object / Array / Bool / Null (dedup via `b3`) or String / Number (dedup via `ocv`/`ocf`). | -| `MIN(col)` / `MAX(col)` | `eql_v2` ORE/OPE aggregates | `ore`, `ope`, **or** ste_vec-extracted String / Number node | Requires a node that emits `ocv` / `ocf` (or a sibling `ore` / `ope` index). | +| `COUNT(DISTINCT col)` | deterministic dedup | An extracted node that emits `b3`, `ocv`, or `ocf` (or a `unique` / `ore` index on the outer column) | A ste_vec-extracted leaf dedups via `b3` (Object / Array / Bool / Null) or `ocv` / `ocf` (String / Number). `ope` is never emitted by ste_vec extraction; it only applies to the outer column. | +| `MIN(col)` / `MAX(col)` | `eql_v2` ORE/OPE aggregates | A ste_vec-extracted String / Number node (`ocv` / `ocf`), **or** a sibling `ore` / `ope` index on the outer column | ste_vec extraction can only produce `ocv` / `ocf` ordering terms. Whole-column ordering uses the outer-column `ore` or `ope` index. | Additionally, `eql_v2.jsonb_array`, `eql_v2.jsonb_contains`, and `eql_v2.jsonb_contained_by` are EQL helpers (not automatic rewrites) used when building **GIN-indexed** containment queries. See [GIN Indexes for JSONB Containment](./database-indexes.md#gin-indexes-for-jsonb-containment) for the full setup. diff --git a/src/ope_cllw_u64_65/compare.sql b/src/ope_cllw_u64_65/compare.sql index b799f9df..47feae74 100644 --- a/src/ope_cllw_u64_65/compare.sql +++ b/src/ope_cllw_u64_65/compare.sql @@ -3,19 +3,19 @@ -- REQUIRE: src/ope_cllw_u64_65/functions.sql ---! @brief Compare two encrypted values using CLWW OPE index terms +--! @brief Compare two encrypted values using CLLW OPE index terms --! --! Performs a three-way comparison (returns -1/0/1) of encrypted values using ---! their fixed-width CLWW OPE ciphertext index terms. Used internally by range +--! their fixed-width CLLW OPE ciphertext index terms. Used internally by range --! operators (<, <=, >, >=) for order-preserving comparisons without decryption. --! ---! @param a eql_v2_encrypted First encrypted value to compare ---! @param b eql_v2_encrypted Second encrypted value to compare +--! @param a eql_v2_encrypted First encrypted value to compare (NOT NULL — function is STRICT) +--! @param b eql_v2_encrypted Second encrypted value to compare (NOT NULL — function is STRICT) --! @return Integer -1 if a < b, 0 if a = b, 1 if a > b --! ---! @note NULL values are sorted before non-NULL values +--! @note Declared STRICT, so NULL inputs short-circuit to NULL before the body runs. --! @note OPE ciphertexts compare via standard lexicographic bytea ordering — ---! no custom per-byte protocol required (unlike the ORE CLWW variants) +--! no custom per-byte protocol required (unlike the ORE CLLW variants). --! --! @see eql_v2.ope_cllw_u64_65 --! @see eql_v2.has_ope_cllw_u64_65 @@ -27,18 +27,6 @@ AS $$ a_term eql_v2.ope_cllw_u64_65; b_term eql_v2.ope_cllw_u64_65; BEGIN - IF a IS NULL AND b IS NULL THEN - RETURN 0; - END IF; - - IF a IS NULL THEN - RETURN -1; - END IF; - - IF b IS NULL THEN - RETURN 1; - END IF; - IF eql_v2.has_ope_cllw_u64_65(a) THEN a_term := eql_v2.ope_cllw_u64_65(a); END IF; diff --git a/src/ope_cllw_u64_65/functions.sql b/src/ope_cllw_u64_65/functions.sql index e06ddd9b..58c2aebf 100644 --- a/src/ope_cllw_u64_65/functions.sql +++ b/src/ope_cllw_u64_65/functions.sql @@ -3,13 +3,13 @@ -- REQUIRE: src/ope_cllw_u64_65/types.sql ---! @brief Extract CLWW OPE index term from JSONB payload +--! @brief Extract CLLW OPE index term from JSONB payload --! ---! Extracts the fixed-width CLWW OPE ciphertext from the 'opf' field of an +--! Extracts the fixed-width CLLW OPE ciphertext from the 'opf' field of an --! encrypted data payload. Used internally for range query comparisons. --! --! @param jsonb containing encrypted EQL payload ---! @return eql_v2.ope_cllw_u64_65 CLWW OPE ciphertext +--! @return eql_v2.ope_cllw_u64_65 CLLW OPE ciphertext --! @throws Exception if 'opf' field is missing when ope index is expected --! --! @see eql_v2.has_ope_cllw_u64_65 @@ -19,10 +19,6 @@ CREATE FUNCTION eql_v2.ope_cllw_u64_65(val jsonb) IMMUTABLE STRICT PARALLEL SAFE AS $$ BEGIN - IF val IS NULL THEN - RETURN NULL; - END IF; - IF NOT (eql_v2.has_ope_cllw_u64_65(val)) THEN RAISE 'Expected a ope_cllw_u64_65 index (opf) value in json: %', val; END IF; @@ -32,29 +28,27 @@ AS $$ $$ LANGUAGE plpgsql; ---! @brief Extract CLWW OPE index term from encrypted column value +--! @brief Extract CLLW OPE index term from encrypted column value --! ---! Extracts the fixed-width CLWW OPE ciphertext from an encrypted column value +--! Extracts the fixed-width CLLW OPE ciphertext from an encrypted column value --! by accessing its underlying JSONB data field. --! --! @param eql_v2_encrypted Encrypted column value ---! @return eql_v2.ope_cllw_u64_65 CLWW OPE ciphertext +--! @return eql_v2.ope_cllw_u64_65 CLLW OPE ciphertext --! --! @see eql_v2.ope_cllw_u64_65(jsonb) CREATE FUNCTION eql_v2.ope_cllw_u64_65(val eql_v2_encrypted) RETURNS eql_v2.ope_cllw_u64_65 IMMUTABLE STRICT PARALLEL SAFE AS $$ - BEGIN - RETURN (SELECT eql_v2.ope_cllw_u64_65(val.data)); - END; -$$ LANGUAGE plpgsql; + SELECT eql_v2.ope_cllw_u64_65(val.data); +$$ LANGUAGE sql; ---! @brief Check if JSONB payload contains CLWW OPE index term +--! @brief Check if JSONB payload contains CLLW OPE index term --! --! Tests whether the encrypted data payload includes an 'opf' field, ---! indicating a fixed-width CLWW OPE ciphertext is available for range queries. +--! indicating a fixed-width CLLW OPE ciphertext is available for range queries. --! --! @param jsonb containing encrypted EQL payload --! @return Boolean True if 'opf' field is present and non-null @@ -64,26 +58,22 @@ CREATE FUNCTION eql_v2.has_ope_cllw_u64_65(val jsonb) RETURNS boolean IMMUTABLE STRICT PARALLEL SAFE AS $$ - BEGIN - RETURN val ->> 'opf' IS NOT NULL; - END; -$$ LANGUAGE plpgsql; + SELECT val ->> 'opf' IS NOT NULL; +$$ LANGUAGE sql; ---! @brief Check if encrypted column value contains CLWW OPE index term +--! @brief Check if encrypted column value contains CLLW OPE index term --! ---! Tests whether an encrypted column value includes a fixed-width CLWW OPE +--! Tests whether an encrypted column value includes a fixed-width CLLW OPE --! ciphertext by checking its underlying JSONB data field. --! --! @param eql_v2_encrypted Encrypted column value ---! @return Boolean True if CLWW OPE ciphertext is present +--! @return Boolean True if CLLW OPE ciphertext is present --! --! @see eql_v2.has_ope_cllw_u64_65(jsonb) CREATE FUNCTION eql_v2.has_ope_cllw_u64_65(val eql_v2_encrypted) RETURNS boolean IMMUTABLE STRICT PARALLEL SAFE AS $$ - BEGIN - RETURN eql_v2.has_ope_cllw_u64_65(val.data); - END; -$$ LANGUAGE plpgsql; + SELECT eql_v2.has_ope_cllw_u64_65(val.data); +$$ LANGUAGE sql; diff --git a/src/ope_cllw_u64_65/types.sql b/src/ope_cllw_u64_65/types.sql index 8d9226f3..9a7424bb 100644 --- a/src/ope_cllw_u64_65/types.sql +++ b/src/ope_cllw_u64_65/types.sql @@ -1,10 +1,10 @@ -- REQUIRE: src/schema.sql ---! @brief CLWW OPE index term type for fixed-width numeric range queries +--! @brief CLLW OPE index term type for fixed-width numeric range queries --! ---! Composite type for CLWW (Chenette, Lewi, Weis, Wu) Order-Preserving Encryption ---! over 64-bit integers. Ciphertexts are 65 bytes (8 bytes per plaintext bit plus ---! one reserved carry byte). +--! Composite type for CLLW (Chenette, Lewi, Weis, Wu) Order-Preserving Encryption +--! over 64-bit integers. Ciphertexts are 65 bytes (8 bytes per plaintext byte, +--! plus one reserved carry byte). --! --! Ciphertexts compare with **standard lexicographic byte ordering** — unlike --! the ORE variants there is no custom per-byte compare protocol. The ciphertext diff --git a/src/ope_cllw_var_8/compare.sql b/src/ope_cllw_var_8/compare.sql index 7d2be1a4..18e4983c 100644 --- a/src/ope_cllw_var_8/compare.sql +++ b/src/ope_cllw_var_8/compare.sql @@ -3,20 +3,20 @@ -- REQUIRE: src/ope_cllw_var_8/functions.sql ---! @brief Compare two encrypted values using variable-width CLWW OPE index terms +--! @brief Compare two encrypted values using variable-width CLLW OPE index terms --! --! Performs a three-way comparison (returns -1/0/1) of encrypted values using ---! their variable-width CLWW OPE ciphertext index terms. Used internally by +--! their variable-width CLLW OPE ciphertext index terms. Used internally by --! range operators (<, <=, >, >=) for order-preserving comparisons without --! decryption. --! ---! @param a eql_v2_encrypted First encrypted value to compare ---! @param b eql_v2_encrypted Second encrypted value to compare +--! @param a eql_v2_encrypted First encrypted value to compare (NOT NULL — function is STRICT) +--! @param b eql_v2_encrypted Second encrypted value to compare (NOT NULL — function is STRICT) --! @return Integer -1 if a < b, 0 if a = b, 1 if a > b --! ---! @note NULL values are sorted before non-NULL values +--! @note Declared STRICT, so NULL inputs short-circuit to NULL before the body runs. --! @note OPE ciphertexts compare via standard lexicographic bytea ordering — ---! bytea compare handles variable-length inputs (shorter prefix is less) +--! bytea compare handles variable-length inputs (shorter prefix is less). --! --! @see eql_v2.ope_cllw_var_8 --! @see eql_v2.has_ope_cllw_var_8 @@ -28,18 +28,6 @@ AS $$ a_term eql_v2.ope_cllw_var_8; b_term eql_v2.ope_cllw_var_8; BEGIN - IF a IS NULL AND b IS NULL THEN - RETURN 0; - END IF; - - IF a IS NULL THEN - RETURN -1; - END IF; - - IF b IS NULL THEN - RETURN 1; - END IF; - IF eql_v2.has_ope_cllw_var_8(a) THEN a_term := eql_v2.ope_cllw_var_8(a); END IF; diff --git a/src/ope_cllw_var_8/functions.sql b/src/ope_cllw_var_8/functions.sql index f3b7c407..feea559c 100644 --- a/src/ope_cllw_var_8/functions.sql +++ b/src/ope_cllw_var_8/functions.sql @@ -3,13 +3,13 @@ -- REQUIRE: src/ope_cllw_var_8/types.sql ---! @brief Extract variable-width CLWW OPE index term from JSONB payload +--! @brief Extract variable-width CLLW OPE index term from JSONB payload --! ---! Extracts the variable-width CLWW OPE ciphertext from the 'opv' field of an +--! Extracts the variable-width CLLW OPE ciphertext from the 'opv' field of an --! encrypted data payload. Used internally for range query comparisons. --! --! @param jsonb containing encrypted EQL payload ---! @return eql_v2.ope_cllw_var_8 Variable-width CLWW OPE ciphertext +--! @return eql_v2.ope_cllw_var_8 Variable-width CLLW OPE ciphertext --! @throws Exception if 'opv' field is missing when ope index is expected --! --! @see eql_v2.has_ope_cllw_var_8 @@ -19,10 +19,6 @@ CREATE FUNCTION eql_v2.ope_cllw_var_8(val jsonb) IMMUTABLE STRICT PARALLEL SAFE AS $$ BEGIN - IF val IS NULL THEN - RETURN NULL; - END IF; - IF NOT (eql_v2.has_ope_cllw_var_8(val)) THEN RAISE 'Expected a ope_cllw_var_8 index (opv) value in json: %', val; END IF; @@ -32,29 +28,27 @@ AS $$ $$ LANGUAGE plpgsql; ---! @brief Extract variable-width CLWW OPE index term from encrypted column value +--! @brief Extract variable-width CLLW OPE index term from encrypted column value --! ---! Extracts the variable-width CLWW OPE ciphertext from an encrypted column value +--! Extracts the variable-width CLLW OPE ciphertext from an encrypted column value --! by accessing its underlying JSONB data field. --! --! @param eql_v2_encrypted Encrypted column value ---! @return eql_v2.ope_cllw_var_8 Variable-width CLWW OPE ciphertext +--! @return eql_v2.ope_cllw_var_8 Variable-width CLLW OPE ciphertext --! --! @see eql_v2.ope_cllw_var_8(jsonb) CREATE FUNCTION eql_v2.ope_cllw_var_8(val eql_v2_encrypted) RETURNS eql_v2.ope_cllw_var_8 IMMUTABLE STRICT PARALLEL SAFE AS $$ - BEGIN - RETURN (SELECT eql_v2.ope_cllw_var_8(val.data)); - END; -$$ LANGUAGE plpgsql; + SELECT eql_v2.ope_cllw_var_8(val.data); +$$ LANGUAGE sql; ---! @brief Check if JSONB payload contains variable-width CLWW OPE index term +--! @brief Check if JSONB payload contains variable-width CLLW OPE index term --! --! Tests whether the encrypted data payload includes an 'opv' field, ---! indicating a variable-width CLWW OPE ciphertext is available for range queries. +--! indicating a variable-width CLLW OPE ciphertext is available for range queries. --! --! @param jsonb containing encrypted EQL payload --! @return Boolean True if 'opv' field is present and non-null @@ -64,26 +58,22 @@ CREATE FUNCTION eql_v2.has_ope_cllw_var_8(val jsonb) RETURNS boolean IMMUTABLE STRICT PARALLEL SAFE AS $$ - BEGIN - RETURN val ->> 'opv' IS NOT NULL; - END; -$$ LANGUAGE plpgsql; + SELECT val ->> 'opv' IS NOT NULL; +$$ LANGUAGE sql; ---! @brief Check if encrypted column value contains variable-width CLWW OPE index term +--! @brief Check if encrypted column value contains variable-width CLLW OPE index term --! ---! Tests whether an encrypted column value includes a variable-width CLWW OPE +--! Tests whether an encrypted column value includes a variable-width CLLW OPE --! ciphertext by checking its underlying JSONB data field. --! --! @param eql_v2_encrypted Encrypted column value ---! @return Boolean True if variable-width CLWW OPE ciphertext is present +--! @return Boolean True if variable-width CLLW OPE ciphertext is present --! --! @see eql_v2.has_ope_cllw_var_8(jsonb) CREATE FUNCTION eql_v2.has_ope_cllw_var_8(val eql_v2_encrypted) RETURNS boolean IMMUTABLE STRICT PARALLEL SAFE AS $$ - BEGIN - RETURN eql_v2.has_ope_cllw_var_8(val.data); - END; -$$ LANGUAGE plpgsql; + SELECT eql_v2.has_ope_cllw_var_8(val.data); +$$ LANGUAGE sql; diff --git a/src/ope_cllw_var_8/types.sql b/src/ope_cllw_var_8/types.sql index b730e730..4f15189f 100644 --- a/src/ope_cllw_var_8/types.sql +++ b/src/ope_cllw_var_8/types.sql @@ -1,8 +1,8 @@ -- REQUIRE: src/schema.sql ---! @brief CLWW OPE index term type for variable-width range queries +--! @brief CLLW OPE index term type for variable-width range queries --! ---! Composite type for variable-width CLWW (Chenette, Lewi, Weis, Wu) +--! Composite type for variable-width CLLW (Chenette, Lewi, Weis, Wu) --! Order-Preserving Encryption. Unlike ope_cllw_u64_65, supports --! variable-length ciphertexts (strings / byte slices). Ciphertext length is --! `8 * plaintext_bytes + 1` (one carry byte + 8 bytes per plaintext byte). diff --git a/src/operators/order_by.sql b/src/operators/order_by.sql index 5d9959f6..6681abfd 100644 --- a/src/operators/order_by.sql +++ b/src/operators/order_by.sql @@ -36,7 +36,7 @@ $$ LANGUAGE plpgsql; --! @brief Extract OPE ciphertext bytes for ordering encrypted values --! ---! Returns the raw CLWW Order-Preserving Encryption ciphertext as `bytea` so +--! Returns the raw CLLW Order-Preserving Encryption ciphertext as `bytea` so --! it can be used as an order key. OPE ciphertexts compare with standard --! lexicographic byte ordering, so the returned bytea can be ordered directly --! with `<`, `=`, `>` (no custom protocol required). @@ -55,20 +55,12 @@ $$ LANGUAGE plpgsql; CREATE FUNCTION eql_v2.order_by_ope(a eql_v2_encrypted) RETURNS bytea IMMUTABLE STRICT PARALLEL SAFE - SET search_path = pg_catalog, extensions, public AS $$ - BEGIN - IF eql_v2.has_ope_cllw_u64_65(a) THEN - RETURN (eql_v2.ope_cllw_u64_65(a)).bytes; - END IF; - - IF eql_v2.has_ope_cllw_var_8(a) THEN - RETURN (eql_v2.ope_cllw_var_8(a)).bytes; - END IF; - - RETURN NULL; + SELECT CASE + WHEN eql_v2.has_ope_cllw_u64_65(a) THEN (eql_v2.ope_cllw_u64_65(a)).bytes + WHEN eql_v2.has_ope_cllw_var_8(a) THEN (eql_v2.ope_cllw_var_8(a)).bytes END; -$$ LANGUAGE plpgsql; +$$ LANGUAGE sql; diff --git a/tests/sqlx/tests/ope_tests.rs b/tests/sqlx/tests/ope_tests.rs index 819e3759..42ec4681 100644 --- a/tests/sqlx/tests/ope_tests.rs +++ b/tests/sqlx/tests/ope_tests.rs @@ -1,4 +1,4 @@ -//! OPE (CLWW Order-Preserving Encryption) tests +//! OPE (CLLW Order-Preserving Encryption) tests //! //! Exercises the `ope_cllw_u64_65` and `ope_cllw_var_8` support wired into //! `eql_v2_encrypted`. Unlike the ORE CLLW variants, OPE ciphertexts compare @@ -133,6 +133,224 @@ async fn encrypted_gt_operator_uses_opf(pool: PgPool) -> Result<()> { Ok(()) } +#[sqlx::test] +async fn encrypted_lte_operator_uses_opf(pool: PgPool) -> Result<()> { + let a = opf_payload(1); + let b = opf_payload(2); + + let lt = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", + a, b + ); + QueryAssertion::new(&pool, <).returns_bool_value(true).await; + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", + a, a + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_gte_operator_uses_opf(pool: PgPool) -> Result<()> { + let a = opf_payload(1); + let b = opf_payload(2); + + let gt = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", + b, a + ); + QueryAssertion::new(&pool, >).returns_bool_value(true).await; + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", + b, b + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_eq_operator_uses_opf(pool: PgPool) -> Result<()> { + let a = opf_payload(1); + let b = opf_payload(2); + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", + a, a + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + + let neq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", + a, b + ); + QueryAssertion::new(&pool, &neq).returns_bool_value(false).await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_neq_operator_uses_opf(pool: PgPool) -> Result<()> { + let a = opf_payload(1); + let b = opf_payload(2); + + let neq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", + a, b + ); + QueryAssertion::new(&pool, &neq).returns_bool_value(true).await; + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", + a, a + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(false).await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_lt_operator_uses_opv(pool: PgPool) -> Result<()> { + let a = opv_payload(&[0xaa, 0x11]); + let b = opv_payload(&[0xbb, 0x11]); + + let sql = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) < eql_v2.to_encrypted('{}'::jsonb)", + a, b + ); + QueryAssertion::new(&pool, &sql) + .returns_bool_value(true) + .await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_gt_operator_uses_opv(pool: PgPool) -> Result<()> { + let a = opv_payload(&[0xaa, 0x11]); + let b = opv_payload(&[0xbb, 0x11]); + + let sql = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) > eql_v2.to_encrypted('{}'::jsonb)", + b, a + ); + QueryAssertion::new(&pool, &sql) + .returns_bool_value(true) + .await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_lte_operator_uses_opv(pool: PgPool) -> Result<()> { + let a = opv_payload(&[0xaa, 0x11]); + let b = opv_payload(&[0xbb, 0x11]); + + let lt = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", + a, b + ); + QueryAssertion::new(&pool, <).returns_bool_value(true).await; + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", + a, a + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_gte_operator_uses_opv(pool: PgPool) -> Result<()> { + let a = opv_payload(&[0xaa, 0x11]); + let b = opv_payload(&[0xbb, 0x11]); + + let gt = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", + b, a + ); + QueryAssertion::new(&pool, >).returns_bool_value(true).await; + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", + b, b + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_eq_operator_uses_opv(pool: PgPool) -> Result<()> { + let a = opv_payload(&[0xaa, 0x11]); + let b = opv_payload(&[0xbb, 0x11]); + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", + a, a + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + + let neq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", + a, b + ); + QueryAssertion::new(&pool, &neq).returns_bool_value(false).await; + Ok(()) +} + +#[sqlx::test] +async fn encrypted_neq_operator_uses_opv(pool: PgPool) -> Result<()> { + let a = opv_payload(&[0xaa, 0x11]); + let b = opv_payload(&[0xbb, 0x11]); + + let neq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", + a, b + ); + QueryAssertion::new(&pool, &neq).returns_bool_value(true).await; + + let eq = format!( + "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", + a, a + ); + QueryAssertion::new(&pool, &eq).returns_bool_value(false).await; + Ok(()) +} + +/// Build the raw 65-byte OPE fixed ciphertext as a hex string (no JSONB +/// wrapper). Mirrors `opf_payload`'s body: a single signal byte at index 8, +/// all other bytes zero. Larger signal → larger ciphertext under lex compare. +fn opf_hex(signal: u8) -> String { + let mut bytes = vec![0u8; 65]; + bytes[8] = signal; + hex::encode(&bytes) +} + +#[sqlx::test] +async fn ore_wins_over_opf_when_both_present(pool: PgPool) -> Result<()> { + // When a row carries both ORE (`ob`) and OPE (`opf`) terms with conflicting + // orderings, eql_v2.compare must dispatch to the ORE branch (it appears + // earlier in the priority chain) — locking in the precedence contract. + // + // Build a value with ORE rank 1 + opf=high(99) and another with ORE rank 2 + // + opf=low(1). ORE-only ordering says (rank 1) < (rank 2). OPE-only + // ordering would say opf=99 > opf=1. compare() must follow ORE → -1. + let opf_high = opf_hex(99); + let opf_low = opf_hex(1); + + // Fixture rows in `ore` have id=N and an `ob` term that orders by N. + let a_sql = format!( + "(create_encrypted_ore_json(1)::jsonb || jsonb_build_object('opf', '{}'))::eql_v2_encrypted", + opf_high + ); + let b_sql = format!( + "(create_encrypted_ore_json(2)::jsonb || jsonb_build_object('opf', '{}'))::eql_v2_encrypted", + opf_low + ); + + let cmp = format!("SELECT eql_v2.compare({}, {})", a_sql, b_sql); + QueryAssertion::new(&pool, &cmp).returns_int_value(-1).await; + Ok(()) +} + #[sqlx::test] async fn compare_opv_short_prefix_sorts_less(pool: PgPool) -> Result<()> { // Shorter ciphertext that is a lex prefix of the longer one. @@ -464,8 +682,8 @@ async fn config_check_rejects_unknown_index(pool: PgPool) -> Result<()> { .expect_err("expected check_indexes to reject unknown index"); let msg = err.to_string(); assert!( - msg.contains("match, ore, ope, unique, ste_vec"), - "expected error to list valid indexes including 'ope'; got: {msg}" + msg.contains("ope") && msg.contains("bogus"), + "expected error to mention the offending 'bogus' index and list 'ope' as valid; got: {msg}" ); Ok(()) } From 35b3341a6d6bd6f6fe5332b54b4f9c49e660ee09 Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 7 May 2026 13:22:23 +1000 Subject: [PATCH 2/7] test(ope): close NULL-handling and aggregate parity gaps with ORE MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds 18 OPE tests covering behaviors that were exercised for ORE but had no OPE equivalent. The ORE suite duplicates each test across three ciphertext variants (block / cllw_u64_8 / cllw_var_8); for OPE we only need one representative per variant per behavior because both opf and opv use the same primitive (bytea lex compare). NULL term in payload (mirrors compare_hmac_with_null_ore_index): - has_opf_false_when_field_is_json_null - has_opv_false_when_field_is_json_null - compare_dispatches_through_null_opf_to_hmac - compare_dispatches_through_null_opv_to_hmac NULL operands at the comparator (codifies STRICT short-circuit): - compare_ope_cllw_u64_65_strict_returns_null_for_null_operand - compare_ope_cllw_var_8_strict_returns_null_for_null_operand ORDER BY NULLS FIRST/LAST × ASC/DESC on opf-encoded data: - order_by_asc_nulls_first_with_opf - order_by_asc_nulls_last_with_opf - order_by_desc_nulls_first_with_opf - order_by_desc_nulls_last_with_opf sort_compare with mixed NULL + opf rows: - sort_compare_asc_puts_nulls_first_with_opf - sort_compare_desc_puts_nulls_last_with_opf MIN / MAX aggregates over OPE-encoded values: - eql_v2_min_with_opf_finds_minimum - eql_v2_max_with_opf_finds_maximum - eql_v2_min_with_opf_null_only_returns_null - eql_v2_max_with_opf_null_only_returns_null BETWEEN range filtering: - between_with_opf_inclusive_bounds - between_with_opv_inclusive_bounds All 47 OPE tests pass; full suite (~430 tests) is green. --- tests/sqlx/tests/ope_tests.rs | 440 ++++++++++++++++++++++++++++++++++ 1 file changed, 440 insertions(+) diff --git a/tests/sqlx/tests/ope_tests.rs b/tests/sqlx/tests/ope_tests.rs index 42ec4681..1706a1da 100644 --- a/tests/sqlx/tests/ope_tests.rs +++ b/tests/sqlx/tests/ope_tests.rs @@ -687,3 +687,443 @@ async fn config_check_rejects_unknown_index(pool: PgPool) -> Result<()> { ); Ok(()) } + +// ========== NULL-handling parity with ORE ========== +// +// ORE has explicit coverage for several NULL scenarios that the OPE surface +// must also satisfy: +// 1. NULL index term in payload (`{"opf": null}` / `{"opv": null}`) — the +// generic `eql_v2.compare` dispatcher must skip the OPE branch and fall +// through to the next available term (mirrors `compare_hmac_with_null_ore_index`). +// 2. NULL operands at the comparator level — the `compare_ope_cllw_*` +// helpers are STRICT, so a NULL operand short-circuits to NULL. +// 3. NULL rows mixed with encrypted rows in ORDER BY / sort_compare / +// MIN / MAX must respect SQL NULL semantics. + +#[sqlx::test] +async fn has_opf_false_when_field_is_json_null(pool: PgPool) -> Result<()> { + // `{"opf": null}` must not trigger OPE detection — same contract as + // `{"ob": null}` for ORE (see compare_hmac_with_null_ore_index). + let sql = r#"SELECT eql_v2.has_ope_cllw_u64_65('{"v":2,"i":{"t":"t","c":"c"},"opf":null}'::jsonb)"#; + QueryAssertion::new(&pool, sql) + .returns_bool_value(false) + .await; + Ok(()) +} + +#[sqlx::test] +async fn has_opv_false_when_field_is_json_null(pool: PgPool) -> Result<()> { + let sql = r#"SELECT eql_v2.has_ope_cllw_var_8('{"v":2,"i":{"t":"t","c":"c"},"opv":null}'::jsonb)"#; + QueryAssertion::new(&pool, sql) + .returns_bool_value(false) + .await; + Ok(()) +} + +#[sqlx::test] +async fn compare_dispatches_through_null_opf_to_hmac(pool: PgPool) -> Result<()> { + // Mirror of `compare_hmac_with_null_ore_index`: when `opf` is JSON null, + // the dispatcher must skip the OPE branch and use the HMAC term instead. + // Without this, two records with `{"opf": null}` would compare equal via + // the OPE branch (both extract to NULL bytes → equal), masking the HMAC + // ordering. + let a = + "('{\"opf\": null}'::jsonb || create_encrypted_json(1, 'hm')::jsonb)::eql_v2_encrypted"; + let b = + "('{\"opf\": null}'::jsonb || create_encrypted_json(2, 'hm')::jsonb)::eql_v2_encrypted"; + let c = + "('{\"opf\": null}'::jsonb || create_encrypted_json(3, 'hm')::jsonb)::eql_v2_encrypted"; + + for (l, r, expected, label) in [ + (a, a, 0, "compare(a, a)"), + (a, b, -1, "compare(a, b)"), + (a, c, -1, "compare(a, c)"), + (b, b, 0, "compare(b, b)"), + (b, a, 1, "compare(b, a)"), + (b, c, -1, "compare(b, c)"), + (c, c, 0, "compare(c, c)"), + (c, b, 1, "compare(c, b)"), + (c, a, 1, "compare(c, a)"), + ] { + let sql = format!("SELECT eql_v2.compare({}, {})", l, r); + let got: i32 = sqlx::query_scalar(&sql).fetch_one(&pool).await?; + assert_eq!(got, expected, "{label} should equal {expected}"); + } + Ok(()) +} + +#[sqlx::test] +async fn compare_dispatches_through_null_opv_to_hmac(pool: PgPool) -> Result<()> { + // Same as the opf variant but for the variable-width term. Establishes + // that {"opv": null} also short-circuits the OPE branch. + let a = + "('{\"opv\": null}'::jsonb || create_encrypted_json(1, 'hm')::jsonb)::eql_v2_encrypted"; + let b = + "('{\"opv\": null}'::jsonb || create_encrypted_json(2, 'hm')::jsonb)::eql_v2_encrypted"; + + let lt: i32 = sqlx::query_scalar(&format!("SELECT eql_v2.compare({}, {})", a, b)) + .fetch_one(&pool) + .await?; + assert_eq!(lt, -1, "compare(a, b) should equal -1"); + + let gt: i32 = sqlx::query_scalar(&format!("SELECT eql_v2.compare({}, {})", b, a)) + .fetch_one(&pool) + .await?; + assert_eq!(gt, 1, "compare(b, a) should equal 1"); + Ok(()) +} + +#[sqlx::test] +async fn compare_ope_cllw_u64_65_strict_returns_null_for_null_operand(pool: PgPool) -> Result<()> { + // The comparator is declared STRICT; the runtime returns NULL before the + // body runs. Codifying this so a future change that drops STRICT won't + // silently change semantics on the sort fast path. + let payload = opf_payload(1); + let lhs_null = format!( + "SELECT eql_v2.compare_ope_cllw_u64_65(NULL, eql_v2.to_encrypted('{}'::jsonb))", + payload + ); + let result: Option = sqlx::query_scalar(&lhs_null).fetch_one(&pool).await?; + assert!(result.is_none(), "compare(NULL, x) should return NULL"); + + let rhs_null = format!( + "SELECT eql_v2.compare_ope_cllw_u64_65(eql_v2.to_encrypted('{}'::jsonb), NULL)", + payload + ); + let result: Option = sqlx::query_scalar(&rhs_null).fetch_one(&pool).await?; + assert!(result.is_none(), "compare(x, NULL) should return NULL"); + Ok(()) +} + +#[sqlx::test] +async fn compare_ope_cllw_var_8_strict_returns_null_for_null_operand(pool: PgPool) -> Result<()> { + let payload = opv_payload(&[0xaa, 0x11]); + let sql = format!( + "SELECT eql_v2.compare_ope_cllw_var_8(NULL, eql_v2.to_encrypted('{}'::jsonb))", + payload + ); + let result: Option = sqlx::query_scalar(&sql).fetch_one(&pool).await?; + assert!(result.is_none(), "compare(NULL, x) should return NULL"); + Ok(()) +} + +// ========== ORDER BY NULLS FIRST/LAST with opf-encoded data ========== +// +// Fixture layout (all four tests use the same shape): +// id=1: NULL +// id=2: opf payload with signal byte = 42 (largest non-NULL) +// id=3: opf payload with signal byte = 3 (smallest non-NULL) +// id=4: NULL +// +// Mirrors `order_by_null_data.sql` for the ORE side. + +async fn install_opf_null_fixture(tx: &mut sqlx::Transaction<'_, sqlx::Postgres>) -> Result<()> { + sqlx::query( + "CREATE TABLE encrypted_opf_nulls( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + e eql_v2_encrypted + )", + ) + .execute(&mut **tx) + .await?; + + sqlx::query("INSERT INTO encrypted_opf_nulls(e) VALUES (NULL)") + .execute(&mut **tx) + .await?; + sqlx::query(&format!( + "INSERT INTO encrypted_opf_nulls(e) VALUES (eql_v2.to_encrypted('{}'::jsonb))", + opf_payload(42) + )) + .execute(&mut **tx) + .await?; + sqlx::query(&format!( + "INSERT INTO encrypted_opf_nulls(e) VALUES (eql_v2.to_encrypted('{}'::jsonb))", + opf_payload(3) + )) + .execute(&mut **tx) + .await?; + sqlx::query("INSERT INTO encrypted_opf_nulls(e) VALUES (NULL)") + .execute(&mut **tx) + .await?; + Ok(()) +} + +#[sqlx::test] +async fn order_by_asc_nulls_first_with_opf(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let row = sqlx::query("SELECT id FROM encrypted_opf_nulls ORDER BY e ASC NULLS FIRST, id") + .fetch_one(&mut *tx) + .await?; + let first_id: i64 = row.try_get(0)?; + assert_eq!( + first_id, 1, + "ASC NULLS FIRST + tiebreak by id should put id=1 first" + ); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn order_by_asc_nulls_last_with_opf(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let row = sqlx::query("SELECT id FROM encrypted_opf_nulls ORDER BY e ASC NULLS LAST") + .fetch_one(&mut *tx) + .await?; + let first_id: i64 = row.try_get(0)?; + assert_eq!( + first_id, 3, + "ASC NULLS LAST should return smallest non-NULL (id=3, opf signal=3) first" + ); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn order_by_desc_nulls_first_with_opf(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let row = sqlx::query("SELECT id FROM encrypted_opf_nulls ORDER BY e DESC NULLS FIRST, id") + .fetch_one(&mut *tx) + .await?; + let first_id: i64 = row.try_get(0)?; + assert_eq!( + first_id, 1, + "DESC NULLS FIRST + tiebreak by id should put id=1 first" + ); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn order_by_desc_nulls_last_with_opf(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let row = sqlx::query("SELECT id FROM encrypted_opf_nulls ORDER BY e DESC NULLS LAST") + .fetch_one(&mut *tx) + .await?; + let first_id: i64 = row.try_get(0)?; + assert_eq!( + first_id, 2, + "DESC NULLS LAST should return largest non-NULL (id=2, opf signal=42) first" + ); + tx.rollback().await?; + Ok(()) +} + +// ========== sort_compare with NULL operands ========== + +#[sqlx::test] +async fn sort_compare_asc_puts_nulls_first_with_opf(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let rows = sqlx::query( + "SELECT id FROM eql_v2.sort_compare( + (SELECT array_agg(id ORDER BY id) FROM encrypted_opf_nulls), + (SELECT array_agg(e ORDER BY id) FROM encrypted_opf_nulls), + 'ASC' + )", + ) + .fetch_all(&mut *tx) + .await?; + let ids: Vec = rows.iter().map(|r| r.try_get(0).unwrap()).collect(); + let mut null_ids = ids[..2].to_vec(); + null_ids.sort_unstable(); + + assert_eq!(rows.len(), 4, "should return all 4 rows"); + assert_eq!(null_ids, vec![1i64, 4], "NULL rows should sort first"); + assert_eq!(ids[2], 3, "smallest non-NULL (signal=3) should follow NULLs"); + assert_eq!(ids[3], 2, "largest non-NULL (signal=42) should sort last"); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn sort_compare_desc_puts_nulls_last_with_opf(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let rows = sqlx::query( + "SELECT id FROM eql_v2.sort_compare( + (SELECT array_agg(id ORDER BY id) FROM encrypted_opf_nulls), + (SELECT array_agg(e ORDER BY id) FROM encrypted_opf_nulls), + 'DESC' + )", + ) + .fetch_all(&mut *tx) + .await?; + let ids: Vec = rows.iter().map(|r| r.try_get(0).unwrap()).collect(); + let mut null_ids = ids[2..].to_vec(); + null_ids.sort_unstable(); + + assert_eq!(rows.len(), 4, "should return all 4 rows"); + assert_eq!(ids[0], 2, "largest non-NULL (signal=42) should sort first"); + assert_eq!(ids[1], 3, "smaller non-NULL (signal=3) should sort second"); + assert_eq!(null_ids, vec![1i64, 4], "NULL rows should sort last"); + tx.rollback().await?; + Ok(()) +} + +// ========== MIN / MAX aggregates over OPE-encoded values ========== +// +// `eql_v2.min` / `eql_v2.max` use `<` / `>`, which dispatch through +// `eql_v2.compare`, so OPE-encoded values must aggregate correctly without +// any aggregate-side changes. + +#[sqlx::test] +async fn eql_v2_min_with_opf_finds_minimum(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + // Smallest non-NULL signal=3 lives at id=3. + let actual: String = + sqlx::query_scalar("SELECT eql_v2.min(e)::text FROM encrypted_opf_nulls") + .fetch_one(&mut *tx) + .await?; + let expected: String = + sqlx::query_scalar("SELECT e::text FROM encrypted_opf_nulls WHERE id = 3") + .fetch_one(&mut *tx) + .await?; + + assert_eq!( + actual, expected, + "eql_v2.min should return the opf row with the smallest signal byte" + ); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn eql_v2_max_with_opf_finds_maximum(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + // Largest non-NULL signal=42 lives at id=2. + let actual: String = + sqlx::query_scalar("SELECT eql_v2.max(e)::text FROM encrypted_opf_nulls") + .fetch_one(&mut *tx) + .await?; + let expected: String = + sqlx::query_scalar("SELECT e::text FROM encrypted_opf_nulls WHERE id = 2") + .fetch_one(&mut *tx) + .await?; + + assert_eq!( + actual, expected, + "eql_v2.max should return the opf row with the largest signal byte" + ); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn eql_v2_min_with_opf_null_only_returns_null(pool: PgPool) -> Result<()> { + // Mirrors `eql_v2_min_with_null_values` for ORE: aggregate over a NULL-only + // selection must return NULL (the STRICT state-transition function never + // runs). + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let result: Option = sqlx::query_scalar( + "SELECT eql_v2.min(e)::text FROM encrypted_opf_nulls WHERE e IS NULL", + ) + .fetch_one(&mut *tx) + .await?; + assert!(result.is_none(), "eql_v2.min over NULL-only should be NULL"); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn eql_v2_max_with_opf_null_only_returns_null(pool: PgPool) -> Result<()> { + let mut tx = pool.begin().await?; + install_opf_null_fixture(&mut tx).await?; + + let result: Option = sqlx::query_scalar( + "SELECT eql_v2.max(e)::text FROM encrypted_opf_nulls WHERE e IS NULL", + ) + .fetch_one(&mut *tx) + .await?; + assert!(result.is_none(), "eql_v2.max over NULL-only should be NULL"); + tx.rollback().await?; + Ok(()) +} + +// ========== BETWEEN with OPE-encoded data ========== +// +// BETWEEN expands to `lo <= x AND x <= hi`, so this exercises both `<=` +// and `>=` dispatching through compare. + +#[sqlx::test] +async fn between_with_opf_inclusive_bounds(pool: PgPool) -> Result<()> { + // signals 1 < 3 < 5 < 7 < 9; BETWEEN 3 AND 7 should include 3, 5, 7. + let mut tx = pool.begin().await?; + sqlx::query( + "CREATE TABLE encrypted_opf_between( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + e eql_v2_encrypted + )", + ) + .execute(&mut *tx) + .await?; + for signal in [1u8, 3, 5, 7, 9] { + sqlx::query(&format!( + "INSERT INTO encrypted_opf_between(e) VALUES (eql_v2.to_encrypted('{}'::jsonb))", + opf_payload(signal) + )) + .execute(&mut *tx) + .await?; + } + + let lo = opf_payload(3); + let hi = opf_payload(7); + let sql = format!( + "SELECT count(*)::bigint FROM encrypted_opf_between + WHERE e BETWEEN eql_v2.to_encrypted('{}'::jsonb) AND eql_v2.to_encrypted('{}'::jsonb)", + lo, hi + ); + let count: i64 = sqlx::query_scalar(&sql).fetch_one(&mut *tx).await?; + assert_eq!(count, 3, "BETWEEN 3 AND 7 should match signals 3, 5, 7"); + tx.rollback().await?; + Ok(()) +} + +#[sqlx::test] +async fn between_with_opv_inclusive_bounds(pool: PgPool) -> Result<()> { + // Variable-width OPE: two-byte ciphertexts compared by bytea lex order. + let mut tx = pool.begin().await?; + sqlx::query( + "CREATE TABLE encrypted_opv_between( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + e eql_v2_encrypted + )", + ) + .execute(&mut *tx) + .await?; + for first_byte in [0x10u8, 0x30, 0x50, 0x70, 0x90] { + sqlx::query(&format!( + "INSERT INTO encrypted_opv_between(e) VALUES (eql_v2.to_encrypted('{}'::jsonb))", + opv_payload(&[first_byte, 0x00]) + )) + .execute(&mut *tx) + .await?; + } + + let lo = opv_payload(&[0x30, 0x00]); + let hi = opv_payload(&[0x70, 0x00]); + let sql = format!( + "SELECT count(*)::bigint FROM encrypted_opv_between + WHERE e BETWEEN eql_v2.to_encrypted('{}'::jsonb) AND eql_v2.to_encrypted('{}'::jsonb)", + lo, hi + ); + let count: i64 = sqlx::query_scalar(&sql).fetch_one(&mut *tx).await?; + assert_eq!(count, 3, "BETWEEN 0x30 AND 0x70 should match 0x30, 0x50, 0x70"); + tx.rollback().await?; + Ok(()) +} From ffb45d05d0874cbbc317e4c5f3e917a093c2a2a9 Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 7 May 2026 13:25:46 +1000 Subject: [PATCH 3/7] docs(ope): list ope as valid outer-column index for COUNT(DISTINCT) The earlier sql-support clarification dropped `ope` from the outer-column index list for COUNT(DISTINCT col). OPE ciphertexts are deterministic within a column key, so an outer-column `ope` index supports DISTINCT semantics just like `unique` and `ore`. Restoring it for parity with the MIN/MAX row. --- docs/reference/sql-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/sql-support.md b/docs/reference/sql-support.md index fc1f25e0..ea5bd930 100644 --- a/docs/reference/sql-support.md +++ b/docs/reference/sql-support.md @@ -139,7 +139,7 @@ When the `ste_vec` index is configured, CipherStash Proxy rewrites these standar | `jsonb_array_elements(arr)` | `eql_v2.jsonb_array_elements(arr)` | Path must resolve to a JSON array node | Set-returning; yields `eql_v2_encrypted`. | | `jsonb_array_elements_text(arr)` | `eql_v2.jsonb_array_elements_text(arr)` | Path must resolve to a JSON array node | Set-returning; yields ciphertext as `text`. | | `COUNT(col)` | plain `count(*)` | — | No encrypted term required. | -| `COUNT(DISTINCT col)` | deterministic dedup | An extracted node that emits `b3`, `ocv`, or `ocf` (or a `unique` / `ore` index on the outer column) | A ste_vec-extracted leaf dedups via `b3` (Object / Array / Bool / Null) or `ocv` / `ocf` (String / Number). `ope` is never emitted by ste_vec extraction; it only applies to the outer column. | +| `COUNT(DISTINCT col)` | deterministic dedup | An extracted node that emits `b3`, `ocv`, or `ocf` (or a `unique` / `ore` / `ope` index on the outer column) | A ste_vec-extracted leaf dedups via `b3` (Object / Array / Bool / Null) or `ocv` / `ocf` (String / Number). `ope` is never emitted by ste_vec extraction; it only applies to the outer column. | | `MIN(col)` / `MAX(col)` | `eql_v2` ORE/OPE aggregates | A ste_vec-extracted String / Number node (`ocv` / `ocf`), **or** a sibling `ore` / `ope` index on the outer column | ste_vec extraction can only produce `ocv` / `ocf` ordering terms. Whole-column ordering uses the outer-column `ore` or `ope` index. | Additionally, `eql_v2.jsonb_array`, `eql_v2.jsonb_contains`, and `eql_v2.jsonb_contained_by` are EQL helpers (not automatic rewrites) used when building **GIN-indexed** containment queries. See [GIN Indexes for JSONB Containment](./database-indexes.md#gin-indexes-for-jsonb-containment) for the full setup. From e40baa15de02d2ed86226852fcf2e31580cf289d Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 7 May 2026 13:29:59 +1000 Subject: [PATCH 4/7] test(ope): assert compare_ope_cllw_var_8(x, NULL) returns NULL Match the bidirectional NULL coverage of the u64_65 sibling test. Asserting only compare(NULL, x) leaves the right-NULL STRICT path unverified; both directions must short-circuit. --- tests/sqlx/tests/ope_tests.rs | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/tests/sqlx/tests/ope_tests.rs b/tests/sqlx/tests/ope_tests.rs index 1706a1da..4b56ee81 100644 --- a/tests/sqlx/tests/ope_tests.rs +++ b/tests/sqlx/tests/ope_tests.rs @@ -798,12 +798,19 @@ async fn compare_ope_cllw_u64_65_strict_returns_null_for_null_operand(pool: PgPo #[sqlx::test] async fn compare_ope_cllw_var_8_strict_returns_null_for_null_operand(pool: PgPool) -> Result<()> { let payload = opv_payload(&[0xaa, 0x11]); - let sql = format!( + let lhs_null = format!( "SELECT eql_v2.compare_ope_cllw_var_8(NULL, eql_v2.to_encrypted('{}'::jsonb))", payload ); - let result: Option = sqlx::query_scalar(&sql).fetch_one(&pool).await?; + let result: Option = sqlx::query_scalar(&lhs_null).fetch_one(&pool).await?; assert!(result.is_none(), "compare(NULL, x) should return NULL"); + + let rhs_null = format!( + "SELECT eql_v2.compare_ope_cllw_var_8(eql_v2.to_encrypted('{}'::jsonb), NULL)", + payload + ); + let result: Option = sqlx::query_scalar(&rhs_null).fetch_one(&pool).await?; + assert!(result.is_none(), "compare(x, NULL) should return NULL"); Ok(()) } From 01f8120076882ba5c8fbe5c082e332bec00ea6a3 Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 7 May 2026 13:33:17 +1000 Subject: [PATCH 5/7] docs(ope): clarify defensive NULL term branches and fix @param tags Two doc-only fixes flagged in PR #197 review: 1. compare.sql @note for both u64_65 and var_8: the term-NULL branches (a_term IS NULL / b_term IS NULL) are NOT redundant with STRICT. STRICT only short-circuits on NULL function inputs (a, b); the term branches handle a non-NULL eql_v2_encrypted whose payload simply lacks the opf/opv field. Mirrors the defensive pattern in compare_ore_block_u64_8_256. 2. functions.sql @param tags for ope_cllw_u64_65 / ope_cllw_var_8 / has_ope_cllw_u64_65 / has_ope_cllw_var_8 now include the parameter name "val" before the type, matching valid Doxygen syntax (and compare.sql which already names a/b correctly). No code behavior change; mise run docs:validate stays clean. --- src/ope_cllw_u64_65/compare.sql | 8 +++++++- src/ope_cllw_u64_65/functions.sql | 8 ++++---- src/ope_cllw_var_8/compare.sql | 8 +++++++- src/ope_cllw_var_8/functions.sql | 8 ++++---- 4 files changed, 22 insertions(+), 10 deletions(-) diff --git a/src/ope_cllw_u64_65/compare.sql b/src/ope_cllw_u64_65/compare.sql index 47feae74..d9e491b1 100644 --- a/src/ope_cllw_u64_65/compare.sql +++ b/src/ope_cllw_u64_65/compare.sql @@ -13,7 +13,13 @@ --! @param b eql_v2_encrypted Second encrypted value to compare (NOT NULL — function is STRICT) --! @return Integer -1 if a < b, 0 if a = b, 1 if a > b --! ---! @note Declared STRICT, so NULL inputs short-circuit to NULL before the body runs. +--! @note Declared STRICT, so NULL function inputs short-circuit to NULL before +--! the body runs. The internal `a_term IS NULL` / `b_term IS NULL` +--! branches are NOT redundant with STRICT — they handle the case where +--! a non-NULL `eql_v2_encrypted` payload simply lacks the `opf` field +--! (i.e. `has_ope_cllw_u64_65` returned false). A NULL term sorts before +--! a present term, mirroring the defensive pattern used in +--! compare_ore_block_u64_8_256. --! @note OPE ciphertexts compare via standard lexicographic bytea ordering — --! no custom per-byte protocol required (unlike the ORE CLLW variants). --! diff --git a/src/ope_cllw_u64_65/functions.sql b/src/ope_cllw_u64_65/functions.sql index 58c2aebf..8790cde5 100644 --- a/src/ope_cllw_u64_65/functions.sql +++ b/src/ope_cllw_u64_65/functions.sql @@ -8,7 +8,7 @@ --! Extracts the fixed-width CLLW OPE ciphertext from the 'opf' field of an --! encrypted data payload. Used internally for range query comparisons. --! ---! @param jsonb containing encrypted EQL payload +--! @param val jsonb encrypted EQL payload --! @return eql_v2.ope_cllw_u64_65 CLLW OPE ciphertext --! @throws Exception if 'opf' field is missing when ope index is expected --! @@ -33,7 +33,7 @@ $$ LANGUAGE plpgsql; --! Extracts the fixed-width CLLW OPE ciphertext from an encrypted column value --! by accessing its underlying JSONB data field. --! ---! @param eql_v2_encrypted Encrypted column value +--! @param val eql_v2_encrypted Encrypted column value --! @return eql_v2.ope_cllw_u64_65 CLLW OPE ciphertext --! --! @see eql_v2.ope_cllw_u64_65(jsonb) @@ -50,7 +50,7 @@ $$ LANGUAGE sql; --! Tests whether the encrypted data payload includes an 'opf' field, --! indicating a fixed-width CLLW OPE ciphertext is available for range queries. --! ---! @param jsonb containing encrypted EQL payload +--! @param val jsonb encrypted EQL payload --! @return Boolean True if 'opf' field is present and non-null --! --! @see eql_v2.ope_cllw_u64_65 @@ -67,7 +67,7 @@ $$ LANGUAGE sql; --! Tests whether an encrypted column value includes a fixed-width CLLW OPE --! ciphertext by checking its underlying JSONB data field. --! ---! @param eql_v2_encrypted Encrypted column value +--! @param val eql_v2_encrypted Encrypted column value --! @return Boolean True if CLLW OPE ciphertext is present --! --! @see eql_v2.has_ope_cllw_u64_65(jsonb) diff --git a/src/ope_cllw_var_8/compare.sql b/src/ope_cllw_var_8/compare.sql index 18e4983c..6ece2a95 100644 --- a/src/ope_cllw_var_8/compare.sql +++ b/src/ope_cllw_var_8/compare.sql @@ -14,7 +14,13 @@ --! @param b eql_v2_encrypted Second encrypted value to compare (NOT NULL — function is STRICT) --! @return Integer -1 if a < b, 0 if a = b, 1 if a > b --! ---! @note Declared STRICT, so NULL inputs short-circuit to NULL before the body runs. +--! @note Declared STRICT, so NULL function inputs short-circuit to NULL before +--! the body runs. The internal `a_term IS NULL` / `b_term IS NULL` +--! branches are NOT redundant with STRICT — they handle the case where +--! a non-NULL `eql_v2_encrypted` payload simply lacks the `opv` field +--! (i.e. `has_ope_cllw_var_8` returned false). A NULL term sorts before +--! a present term, mirroring the defensive pattern used in +--! compare_ore_block_u64_8_256. --! @note OPE ciphertexts compare via standard lexicographic bytea ordering — --! bytea compare handles variable-length inputs (shorter prefix is less). --! diff --git a/src/ope_cllw_var_8/functions.sql b/src/ope_cllw_var_8/functions.sql index feea559c..938cefdd 100644 --- a/src/ope_cllw_var_8/functions.sql +++ b/src/ope_cllw_var_8/functions.sql @@ -8,7 +8,7 @@ --! Extracts the variable-width CLLW OPE ciphertext from the 'opv' field of an --! encrypted data payload. Used internally for range query comparisons. --! ---! @param jsonb containing encrypted EQL payload +--! @param val jsonb encrypted EQL payload --! @return eql_v2.ope_cllw_var_8 Variable-width CLLW OPE ciphertext --! @throws Exception if 'opv' field is missing when ope index is expected --! @@ -33,7 +33,7 @@ $$ LANGUAGE plpgsql; --! Extracts the variable-width CLLW OPE ciphertext from an encrypted column value --! by accessing its underlying JSONB data field. --! ---! @param eql_v2_encrypted Encrypted column value +--! @param val eql_v2_encrypted Encrypted column value --! @return eql_v2.ope_cllw_var_8 Variable-width CLLW OPE ciphertext --! --! @see eql_v2.ope_cllw_var_8(jsonb) @@ -50,7 +50,7 @@ $$ LANGUAGE sql; --! Tests whether the encrypted data payload includes an 'opv' field, --! indicating a variable-width CLLW OPE ciphertext is available for range queries. --! ---! @param jsonb containing encrypted EQL payload +--! @param val jsonb encrypted EQL payload --! @return Boolean True if 'opv' field is present and non-null --! --! @see eql_v2.ope_cllw_var_8 @@ -67,7 +67,7 @@ $$ LANGUAGE sql; --! Tests whether an encrypted column value includes a variable-width CLLW OPE --! ciphertext by checking its underlying JSONB data field. --! ---! @param eql_v2_encrypted Encrypted column value +--! @param val eql_v2_encrypted Encrypted column value --! @return Boolean True if variable-width CLLW OPE ciphertext is present --! --! @see eql_v2.has_ope_cllw_var_8(jsonb) From 8533b67fbff79e8ef11e236d79673765810123c3 Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 7 May 2026 13:47:09 +1000 Subject: [PATCH 6/7] chore: fmt --- tests/sqlx/tests/ope_tests.rs | 127 +++++++++++++++++++++------------- 1 file changed, 79 insertions(+), 48 deletions(-) diff --git a/tests/sqlx/tests/ope_tests.rs b/tests/sqlx/tests/ope_tests.rs index 4b56ee81..c0b9b6db 100644 --- a/tests/sqlx/tests/ope_tests.rs +++ b/tests/sqlx/tests/ope_tests.rs @@ -142,13 +142,17 @@ async fn encrypted_lte_operator_uses_opf(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", a, b ); - QueryAssertion::new(&pool, <).returns_bool_value(true).await; + QueryAssertion::new(&pool, <) + .returns_bool_value(true) + .await; let eq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", a, a ); - QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(true) + .await; Ok(()) } @@ -161,13 +165,17 @@ async fn encrypted_gte_operator_uses_opf(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", b, a ); - QueryAssertion::new(&pool, >).returns_bool_value(true).await; + QueryAssertion::new(&pool, >) + .returns_bool_value(true) + .await; let eq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", b, b ); - QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(true) + .await; Ok(()) } @@ -180,13 +188,17 @@ async fn encrypted_eq_operator_uses_opf(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", a, a ); - QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(true) + .await; let neq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", a, b ); - QueryAssertion::new(&pool, &neq).returns_bool_value(false).await; + QueryAssertion::new(&pool, &neq) + .returns_bool_value(false) + .await; Ok(()) } @@ -199,13 +211,17 @@ async fn encrypted_neq_operator_uses_opf(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", a, b ); - QueryAssertion::new(&pool, &neq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &neq) + .returns_bool_value(true) + .await; let eq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", a, a ); - QueryAssertion::new(&pool, &eq).returns_bool_value(false).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(false) + .await; Ok(()) } @@ -248,13 +264,17 @@ async fn encrypted_lte_operator_uses_opv(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", a, b ); - QueryAssertion::new(&pool, <).returns_bool_value(true).await; + QueryAssertion::new(&pool, <) + .returns_bool_value(true) + .await; let eq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) <= eql_v2.to_encrypted('{}'::jsonb)", a, a ); - QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(true) + .await; Ok(()) } @@ -267,13 +287,17 @@ async fn encrypted_gte_operator_uses_opv(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", b, a ); - QueryAssertion::new(&pool, >).returns_bool_value(true).await; + QueryAssertion::new(&pool, >) + .returns_bool_value(true) + .await; let eq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) >= eql_v2.to_encrypted('{}'::jsonb)", b, b ); - QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(true) + .await; Ok(()) } @@ -286,13 +310,17 @@ async fn encrypted_eq_operator_uses_opv(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", a, a ); - QueryAssertion::new(&pool, &eq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(true) + .await; let neq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) = eql_v2.to_encrypted('{}'::jsonb)", a, b ); - QueryAssertion::new(&pool, &neq).returns_bool_value(false).await; + QueryAssertion::new(&pool, &neq) + .returns_bool_value(false) + .await; Ok(()) } @@ -305,13 +333,17 @@ async fn encrypted_neq_operator_uses_opv(pool: PgPool) -> Result<()> { "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", a, b ); - QueryAssertion::new(&pool, &neq).returns_bool_value(true).await; + QueryAssertion::new(&pool, &neq) + .returns_bool_value(true) + .await; let eq = format!( "SELECT eql_v2.to_encrypted('{}'::jsonb) <> eql_v2.to_encrypted('{}'::jsonb)", a, a ); - QueryAssertion::new(&pool, &eq).returns_bool_value(false).await; + QueryAssertion::new(&pool, &eq) + .returns_bool_value(false) + .await; Ok(()) } @@ -704,7 +736,8 @@ async fn config_check_rejects_unknown_index(pool: PgPool) -> Result<()> { async fn has_opf_false_when_field_is_json_null(pool: PgPool) -> Result<()> { // `{"opf": null}` must not trigger OPE detection — same contract as // `{"ob": null}` for ORE (see compare_hmac_with_null_ore_index). - let sql = r#"SELECT eql_v2.has_ope_cllw_u64_65('{"v":2,"i":{"t":"t","c":"c"},"opf":null}'::jsonb)"#; + let sql = + r#"SELECT eql_v2.has_ope_cllw_u64_65('{"v":2,"i":{"t":"t","c":"c"},"opf":null}'::jsonb)"#; QueryAssertion::new(&pool, sql) .returns_bool_value(false) .await; @@ -713,7 +746,8 @@ async fn has_opf_false_when_field_is_json_null(pool: PgPool) -> Result<()> { #[sqlx::test] async fn has_opv_false_when_field_is_json_null(pool: PgPool) -> Result<()> { - let sql = r#"SELECT eql_v2.has_ope_cllw_var_8('{"v":2,"i":{"t":"t","c":"c"},"opv":null}'::jsonb)"#; + let sql = + r#"SELECT eql_v2.has_ope_cllw_var_8('{"v":2,"i":{"t":"t","c":"c"},"opv":null}'::jsonb)"#; QueryAssertion::new(&pool, sql) .returns_bool_value(false) .await; @@ -727,12 +761,9 @@ async fn compare_dispatches_through_null_opf_to_hmac(pool: PgPool) -> Result<()> // Without this, two records with `{"opf": null}` would compare equal via // the OPE branch (both extract to NULL bytes → equal), masking the HMAC // ordering. - let a = - "('{\"opf\": null}'::jsonb || create_encrypted_json(1, 'hm')::jsonb)::eql_v2_encrypted"; - let b = - "('{\"opf\": null}'::jsonb || create_encrypted_json(2, 'hm')::jsonb)::eql_v2_encrypted"; - let c = - "('{\"opf\": null}'::jsonb || create_encrypted_json(3, 'hm')::jsonb)::eql_v2_encrypted"; + let a = "('{\"opf\": null}'::jsonb || create_encrypted_json(1, 'hm')::jsonb)::eql_v2_encrypted"; + let b = "('{\"opf\": null}'::jsonb || create_encrypted_json(2, 'hm')::jsonb)::eql_v2_encrypted"; + let c = "('{\"opf\": null}'::jsonb || create_encrypted_json(3, 'hm')::jsonb)::eql_v2_encrypted"; for (l, r, expected, label) in [ (a, a, 0, "compare(a, a)"), @@ -756,10 +787,8 @@ async fn compare_dispatches_through_null_opf_to_hmac(pool: PgPool) -> Result<()> async fn compare_dispatches_through_null_opv_to_hmac(pool: PgPool) -> Result<()> { // Same as the opf variant but for the variable-width term. Establishes // that {"opv": null} also short-circuits the OPE branch. - let a = - "('{\"opv\": null}'::jsonb || create_encrypted_json(1, 'hm')::jsonb)::eql_v2_encrypted"; - let b = - "('{\"opv\": null}'::jsonb || create_encrypted_json(2, 'hm')::jsonb)::eql_v2_encrypted"; + let a = "('{\"opv\": null}'::jsonb || create_encrypted_json(1, 'hm')::jsonb)::eql_v2_encrypted"; + let b = "('{\"opv\": null}'::jsonb || create_encrypted_json(2, 'hm')::jsonb)::eql_v2_encrypted"; let lt: i32 = sqlx::query_scalar(&format!("SELECT eql_v2.compare({}, {})", a, b)) .fetch_one(&pool) @@ -945,7 +974,10 @@ async fn sort_compare_asc_puts_nulls_first_with_opf(pool: PgPool) -> Result<()> assert_eq!(rows.len(), 4, "should return all 4 rows"); assert_eq!(null_ids, vec![1i64, 4], "NULL rows should sort first"); - assert_eq!(ids[2], 3, "smallest non-NULL (signal=3) should follow NULLs"); + assert_eq!( + ids[2], 3, + "smallest non-NULL (signal=3) should follow NULLs" + ); assert_eq!(ids[3], 2, "largest non-NULL (signal=42) should sort last"); tx.rollback().await?; Ok(()) @@ -989,10 +1021,9 @@ async fn eql_v2_min_with_opf_finds_minimum(pool: PgPool) -> Result<()> { install_opf_null_fixture(&mut tx).await?; // Smallest non-NULL signal=3 lives at id=3. - let actual: String = - sqlx::query_scalar("SELECT eql_v2.min(e)::text FROM encrypted_opf_nulls") - .fetch_one(&mut *tx) - .await?; + let actual: String = sqlx::query_scalar("SELECT eql_v2.min(e)::text FROM encrypted_opf_nulls") + .fetch_one(&mut *tx) + .await?; let expected: String = sqlx::query_scalar("SELECT e::text FROM encrypted_opf_nulls WHERE id = 3") .fetch_one(&mut *tx) @@ -1012,10 +1043,9 @@ async fn eql_v2_max_with_opf_finds_maximum(pool: PgPool) -> Result<()> { install_opf_null_fixture(&mut tx).await?; // Largest non-NULL signal=42 lives at id=2. - let actual: String = - sqlx::query_scalar("SELECT eql_v2.max(e)::text FROM encrypted_opf_nulls") - .fetch_one(&mut *tx) - .await?; + let actual: String = sqlx::query_scalar("SELECT eql_v2.max(e)::text FROM encrypted_opf_nulls") + .fetch_one(&mut *tx) + .await?; let expected: String = sqlx::query_scalar("SELECT e::text FROM encrypted_opf_nulls WHERE id = 2") .fetch_one(&mut *tx) @@ -1037,11 +1067,10 @@ async fn eql_v2_min_with_opf_null_only_returns_null(pool: PgPool) -> Result<()> let mut tx = pool.begin().await?; install_opf_null_fixture(&mut tx).await?; - let result: Option = sqlx::query_scalar( - "SELECT eql_v2.min(e)::text FROM encrypted_opf_nulls WHERE e IS NULL", - ) - .fetch_one(&mut *tx) - .await?; + let result: Option = + sqlx::query_scalar("SELECT eql_v2.min(e)::text FROM encrypted_opf_nulls WHERE e IS NULL") + .fetch_one(&mut *tx) + .await?; assert!(result.is_none(), "eql_v2.min over NULL-only should be NULL"); tx.rollback().await?; Ok(()) @@ -1052,11 +1081,10 @@ async fn eql_v2_max_with_opf_null_only_returns_null(pool: PgPool) -> Result<()> let mut tx = pool.begin().await?; install_opf_null_fixture(&mut tx).await?; - let result: Option = sqlx::query_scalar( - "SELECT eql_v2.max(e)::text FROM encrypted_opf_nulls WHERE e IS NULL", - ) - .fetch_one(&mut *tx) - .await?; + let result: Option = + sqlx::query_scalar("SELECT eql_v2.max(e)::text FROM encrypted_opf_nulls WHERE e IS NULL") + .fetch_one(&mut *tx) + .await?; assert!(result.is_none(), "eql_v2.max over NULL-only should be NULL"); tx.rollback().await?; Ok(()) @@ -1130,7 +1158,10 @@ async fn between_with_opv_inclusive_bounds(pool: PgPool) -> Result<()> { lo, hi ); let count: i64 = sqlx::query_scalar(&sql).fetch_one(&mut *tx).await?; - assert_eq!(count, 3, "BETWEEN 0x30 AND 0x70 should match 0x30, 0x50, 0x70"); + assert_eq!( + count, 3, + "BETWEEN 0x30 AND 0x70 should match 0x30, 0x50, 0x70" + ); tx.rollback().await?; Ok(()) } From 4efadbde6f541017414c349db669a190695a32d3 Mon Sep 17 00:00:00 2001 From: James Sadler Date: Thu, 7 May 2026 16:39:40 +1000 Subject: [PATCH 7/7] docs(ope): restore CLWW spelling for the OPE cipher MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per coderdan: Chenette-Lewi-Weis-Wu = CLWW. The earlier change in this PR (CLWW -> CLLW everywhere "for consistency with type names") was wrong-headed — the type names ope_cllw_u64_65 / ope_cllw_var_8 are themselves typos that already shipped, but the cipher abbreviation in prose docs should stay CLWW. Spreading the typo to free-text docs makes the codebase less searchable, not more. Reverts CLLW -> CLWW only in OPE-context references: docs/reference/{index-config,sql-support}.md src/ope_cllw_u64_65/{compare,functions,types}.sql src/ope_cllw_var_8/{compare,functions,types}.sql src/operators/order_by.sql tests/sqlx/tests/ope_tests.rs (file-level docstring) Leaves untouched: - The "ORE CLLW variants" / "CLLW ORE" / "CLLW Order-Revealing" references — those are the genuine CLLW (Copyless Logarithmic Width) ORE cipher used by ore_cllw_u64_8 and ore_cllw_var_8. - Type / function / file names (ope_cllw_*) — out of scope to rename. 47/47 OPE tests still pass; docs:validate clean. --- docs/reference/index-config.md | 2 +- docs/reference/sql-support.md | 2 +- src/ope_cllw_u64_65/compare.sql | 4 ++-- src/ope_cllw_u64_65/functions.sql | 22 +++++++++++----------- src/ope_cllw_u64_65/types.sql | 4 ++-- src/ope_cllw_var_8/compare.sql | 4 ++-- src/ope_cllw_var_8/functions.sql | 22 +++++++++++----------- src/ope_cllw_var_8/types.sql | 4 ++-- src/operators/order_by.sql | 2 +- tests/sqlx/tests/ope_tests.rs | 2 +- 10 files changed, 34 insertions(+), 34 deletions(-) diff --git a/docs/reference/index-config.md b/docs/reference/index-config.md index 094f39de..c5e01746 100644 --- a/docs/reference/index-config.md +++ b/docs/reference/index-config.md @@ -114,7 +114,7 @@ Try to ensure that the string you search for is at least as long as the `tokenLe Both `ore` and `ope` enable the same ordered-comparison surface (`<`, `<=`, `=`, `>`, `>=`, `BETWEEN`, `ORDER BY`, `MIN`/`MAX`). - **`ore`** uses Order-Revealing Encryption (`ore_block_u64_8_256`, payload field `ob`). Ciphertexts compare via a custom per-byte protocol implemented in `eql_v2.compare_ore_block_u64_8_256`. This is the default ordered-search index. -- **`ope`** uses CLLW Order-Preserving Encryption — `ope_cllw_u64_65` (fixed-width, payload field `opf`) for numeric types and `ope_cllw_var_8` (variable-width, payload field `opv`) for text-shaped values. OPE ciphertexts compare with **standard lexicographic byte ordering**, which makes them usable in environments that can only sort `bytea` natively (e.g. some pluggable storage layers without custom comparators). +- **`ope`** uses CLWW Order-Preserving Encryption — `ope_cllw_u64_65` (fixed-width, payload field `opf`) for numeric types and `ope_cllw_var_8` (variable-width, payload field `opv`) for text-shaped values. OPE ciphertexts compare with **standard lexicographic byte ordering**, which makes them usable in environments that can only sort `bytea` natively (e.g. some pluggable storage layers without custom comparators). `eql_v2.compare()` and the `<` / `<=` / `>` / `>=` operators dispatch automatically to whichever ordered terms are present on the encrypted value, so application queries do not change when switching between `ore` and `ope`. diff --git a/docs/reference/sql-support.md b/docs/reference/sql-support.md index ea5bd930..782df23d 100644 --- a/docs/reference/sql-support.md +++ b/docs/reference/sql-support.md @@ -12,7 +12,7 @@ EQL ships five search index kinds that encrypt data in ways that preserve specif | `match` | `bloom_filter` (`bf`) | Substring / token matching via `LIKE` / `ILIKE` | | `ste_vec` | Structured encryption (`sv`) | JSONB containment and JSONB path / field access | -> **`ore` vs `ope`** — both index kinds support the same ordered-comparison surface. `ore` (Order-Revealing Encryption) is the default. `ope` (CLLW Order-Preserving Encryption) is an alternative for environments that need plain lexicographic byte comparison (e.g. pluggable storage that cannot run a custom comparator). On a column configured for `ope`, `eql_v2.compare()` and the `<` / `<=` / `>` / `>=` operators dispatch to OPE terms automatically. +> **`ore` vs `ope`** — both index kinds support the same ordered-comparison surface. `ore` (Order-Revealing Encryption) is the default. `ope` (CLWW Order-Preserving Encryption) is an alternative for environments that need plain lexicographic byte comparison (e.g. pluggable storage that cannot run a custom comparator). On a column configured for `ope`, `eql_v2.compare()` and the `<` / `<=` / `>` / `>=` operators dispatch to OPE terms automatically. Every column must also be registered with `eql_v2.add_column(...)` — that alone gives the column storage and decryption, but none of the operators below will produce results until at least one search index is added for the operation you need. diff --git a/src/ope_cllw_u64_65/compare.sql b/src/ope_cllw_u64_65/compare.sql index d9e491b1..7f3ff789 100644 --- a/src/ope_cllw_u64_65/compare.sql +++ b/src/ope_cllw_u64_65/compare.sql @@ -3,10 +3,10 @@ -- REQUIRE: src/ope_cllw_u64_65/functions.sql ---! @brief Compare two encrypted values using CLLW OPE index terms +--! @brief Compare two encrypted values using CLWW OPE index terms --! --! Performs a three-way comparison (returns -1/0/1) of encrypted values using ---! their fixed-width CLLW OPE ciphertext index terms. Used internally by range +--! their fixed-width CLWW OPE ciphertext index terms. Used internally by range --! operators (<, <=, >, >=) for order-preserving comparisons without decryption. --! --! @param a eql_v2_encrypted First encrypted value to compare (NOT NULL — function is STRICT) diff --git a/src/ope_cllw_u64_65/functions.sql b/src/ope_cllw_u64_65/functions.sql index 8790cde5..20bbca22 100644 --- a/src/ope_cllw_u64_65/functions.sql +++ b/src/ope_cllw_u64_65/functions.sql @@ -3,13 +3,13 @@ -- REQUIRE: src/ope_cllw_u64_65/types.sql ---! @brief Extract CLLW OPE index term from JSONB payload +--! @brief Extract CLWW OPE index term from JSONB payload --! ---! Extracts the fixed-width CLLW OPE ciphertext from the 'opf' field of an +--! Extracts the fixed-width CLWW OPE ciphertext from the 'opf' field of an --! encrypted data payload. Used internally for range query comparisons. --! --! @param val jsonb encrypted EQL payload ---! @return eql_v2.ope_cllw_u64_65 CLLW OPE ciphertext +--! @return eql_v2.ope_cllw_u64_65 CLWW OPE ciphertext --! @throws Exception if 'opf' field is missing when ope index is expected --! --! @see eql_v2.has_ope_cllw_u64_65 @@ -28,13 +28,13 @@ AS $$ $$ LANGUAGE plpgsql; ---! @brief Extract CLLW OPE index term from encrypted column value +--! @brief Extract CLWW OPE index term from encrypted column value --! ---! Extracts the fixed-width CLLW OPE ciphertext from an encrypted column value +--! Extracts the fixed-width CLWW OPE ciphertext from an encrypted column value --! by accessing its underlying JSONB data field. --! --! @param val eql_v2_encrypted Encrypted column value ---! @return eql_v2.ope_cllw_u64_65 CLLW OPE ciphertext +--! @return eql_v2.ope_cllw_u64_65 CLWW OPE ciphertext --! --! @see eql_v2.ope_cllw_u64_65(jsonb) CREATE FUNCTION eql_v2.ope_cllw_u64_65(val eql_v2_encrypted) @@ -45,10 +45,10 @@ AS $$ $$ LANGUAGE sql; ---! @brief Check if JSONB payload contains CLLW OPE index term +--! @brief Check if JSONB payload contains CLWW OPE index term --! --! Tests whether the encrypted data payload includes an 'opf' field, ---! indicating a fixed-width CLLW OPE ciphertext is available for range queries. +--! indicating a fixed-width CLWW OPE ciphertext is available for range queries. --! --! @param val jsonb encrypted EQL payload --! @return Boolean True if 'opf' field is present and non-null @@ -62,13 +62,13 @@ AS $$ $$ LANGUAGE sql; ---! @brief Check if encrypted column value contains CLLW OPE index term +--! @brief Check if encrypted column value contains CLWW OPE index term --! ---! Tests whether an encrypted column value includes a fixed-width CLLW OPE +--! Tests whether an encrypted column value includes a fixed-width CLWW OPE --! ciphertext by checking its underlying JSONB data field. --! --! @param val eql_v2_encrypted Encrypted column value ---! @return Boolean True if CLLW OPE ciphertext is present +--! @return Boolean True if CLWW OPE ciphertext is present --! --! @see eql_v2.has_ope_cllw_u64_65(jsonb) CREATE FUNCTION eql_v2.has_ope_cllw_u64_65(val eql_v2_encrypted) diff --git a/src/ope_cllw_u64_65/types.sql b/src/ope_cllw_u64_65/types.sql index 9a7424bb..177a46fe 100644 --- a/src/ope_cllw_u64_65/types.sql +++ b/src/ope_cllw_u64_65/types.sql @@ -1,8 +1,8 @@ -- REQUIRE: src/schema.sql ---! @brief CLLW OPE index term type for fixed-width numeric range queries +--! @brief CLWW OPE index term type for fixed-width numeric range queries --! ---! Composite type for CLLW (Chenette, Lewi, Weis, Wu) Order-Preserving Encryption +--! Composite type for CLWW (Chenette, Lewi, Weis, Wu) Order-Preserving Encryption --! over 64-bit integers. Ciphertexts are 65 bytes (8 bytes per plaintext byte, --! plus one reserved carry byte). --! diff --git a/src/ope_cllw_var_8/compare.sql b/src/ope_cllw_var_8/compare.sql index 6ece2a95..f98f0701 100644 --- a/src/ope_cllw_var_8/compare.sql +++ b/src/ope_cllw_var_8/compare.sql @@ -3,10 +3,10 @@ -- REQUIRE: src/ope_cllw_var_8/functions.sql ---! @brief Compare two encrypted values using variable-width CLLW OPE index terms +--! @brief Compare two encrypted values using variable-width CLWW OPE index terms --! --! Performs a three-way comparison (returns -1/0/1) of encrypted values using ---! their variable-width CLLW OPE ciphertext index terms. Used internally by +--! their variable-width CLWW OPE ciphertext index terms. Used internally by --! range operators (<, <=, >, >=) for order-preserving comparisons without --! decryption. --! diff --git a/src/ope_cllw_var_8/functions.sql b/src/ope_cllw_var_8/functions.sql index 938cefdd..05a2bcc6 100644 --- a/src/ope_cllw_var_8/functions.sql +++ b/src/ope_cllw_var_8/functions.sql @@ -3,13 +3,13 @@ -- REQUIRE: src/ope_cllw_var_8/types.sql ---! @brief Extract variable-width CLLW OPE index term from JSONB payload +--! @brief Extract variable-width CLWW OPE index term from JSONB payload --! ---! Extracts the variable-width CLLW OPE ciphertext from the 'opv' field of an +--! Extracts the variable-width CLWW OPE ciphertext from the 'opv' field of an --! encrypted data payload. Used internally for range query comparisons. --! --! @param val jsonb encrypted EQL payload ---! @return eql_v2.ope_cllw_var_8 Variable-width CLLW OPE ciphertext +--! @return eql_v2.ope_cllw_var_8 Variable-width CLWW OPE ciphertext --! @throws Exception if 'opv' field is missing when ope index is expected --! --! @see eql_v2.has_ope_cllw_var_8 @@ -28,13 +28,13 @@ AS $$ $$ LANGUAGE plpgsql; ---! @brief Extract variable-width CLLW OPE index term from encrypted column value +--! @brief Extract variable-width CLWW OPE index term from encrypted column value --! ---! Extracts the variable-width CLLW OPE ciphertext from an encrypted column value +--! Extracts the variable-width CLWW OPE ciphertext from an encrypted column value --! by accessing its underlying JSONB data field. --! --! @param val eql_v2_encrypted Encrypted column value ---! @return eql_v2.ope_cllw_var_8 Variable-width CLLW OPE ciphertext +--! @return eql_v2.ope_cllw_var_8 Variable-width CLWW OPE ciphertext --! --! @see eql_v2.ope_cllw_var_8(jsonb) CREATE FUNCTION eql_v2.ope_cllw_var_8(val eql_v2_encrypted) @@ -45,10 +45,10 @@ AS $$ $$ LANGUAGE sql; ---! @brief Check if JSONB payload contains variable-width CLLW OPE index term +--! @brief Check if JSONB payload contains variable-width CLWW OPE index term --! --! Tests whether the encrypted data payload includes an 'opv' field, ---! indicating a variable-width CLLW OPE ciphertext is available for range queries. +--! indicating a variable-width CLWW OPE ciphertext is available for range queries. --! --! @param val jsonb encrypted EQL payload --! @return Boolean True if 'opv' field is present and non-null @@ -62,13 +62,13 @@ AS $$ $$ LANGUAGE sql; ---! @brief Check if encrypted column value contains variable-width CLLW OPE index term +--! @brief Check if encrypted column value contains variable-width CLWW OPE index term --! ---! Tests whether an encrypted column value includes a variable-width CLLW OPE +--! Tests whether an encrypted column value includes a variable-width CLWW OPE --! ciphertext by checking its underlying JSONB data field. --! --! @param val eql_v2_encrypted Encrypted column value ---! @return Boolean True if variable-width CLLW OPE ciphertext is present +--! @return Boolean True if variable-width CLWW OPE ciphertext is present --! --! @see eql_v2.has_ope_cllw_var_8(jsonb) CREATE FUNCTION eql_v2.has_ope_cllw_var_8(val eql_v2_encrypted) diff --git a/src/ope_cllw_var_8/types.sql b/src/ope_cllw_var_8/types.sql index 4f15189f..b730e730 100644 --- a/src/ope_cllw_var_8/types.sql +++ b/src/ope_cllw_var_8/types.sql @@ -1,8 +1,8 @@ -- REQUIRE: src/schema.sql ---! @brief CLLW OPE index term type for variable-width range queries +--! @brief CLWW OPE index term type for variable-width range queries --! ---! Composite type for variable-width CLLW (Chenette, Lewi, Weis, Wu) +--! Composite type for variable-width CLWW (Chenette, Lewi, Weis, Wu) --! Order-Preserving Encryption. Unlike ope_cllw_u64_65, supports --! variable-length ciphertexts (strings / byte slices). Ciphertext length is --! `8 * plaintext_bytes + 1` (one carry byte + 8 bytes per plaintext byte). diff --git a/src/operators/order_by.sql b/src/operators/order_by.sql index 6681abfd..6e85e58c 100644 --- a/src/operators/order_by.sql +++ b/src/operators/order_by.sql @@ -36,7 +36,7 @@ $$ LANGUAGE plpgsql; --! @brief Extract OPE ciphertext bytes for ordering encrypted values --! ---! Returns the raw CLLW Order-Preserving Encryption ciphertext as `bytea` so +--! Returns the raw CLWW Order-Preserving Encryption ciphertext as `bytea` so --! it can be used as an order key. OPE ciphertexts compare with standard --! lexicographic byte ordering, so the returned bytea can be ordered directly --! with `<`, `=`, `>` (no custom protocol required). diff --git a/tests/sqlx/tests/ope_tests.rs b/tests/sqlx/tests/ope_tests.rs index c0b9b6db..0d0e1a87 100644 --- a/tests/sqlx/tests/ope_tests.rs +++ b/tests/sqlx/tests/ope_tests.rs @@ -1,4 +1,4 @@ -//! OPE (CLLW Order-Preserving Encryption) tests +//! OPE (CLWW Order-Preserving Encryption) tests //! //! Exercises the `ope_cllw_u64_65` and `ope_cllw_var_8` support wired into //! `eql_v2_encrypted`. Unlike the ORE CLLW variants, OPE ciphertexts compare