Skip to content

Fix non-deterministic ordering in message polling queries#15

Closed
Copilot wants to merge 4 commits intomainfrom
copilot/fix-non-deterministic-test-failure
Closed

Fix non-deterministic ordering in message polling queries#15
Copilot wants to merge 4 commits intomainfrom
copilot/fix-non-deterministic-test-failure

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 7, 2026

H2 2.4.240 upgrade exposed non-deterministic test failures when messages share identical scheduledAt timestamps. ORDER BY scheduledAt alone permits arbitrary row ordering in this case.

Changes

Query Updates

  • Add secondary id sort to acquireManyOptimistically and selectFirstAvailableWithLock across all adapters
  • Ensures deterministic ordering: ORDER BY scheduledAt, id

Index Optimization

  • Update KindPlusScheduledAtIndex from (pKind, scheduledAt) to (pKind, scheduledAt, id)
  • Eliminates sort operations—queries are now fully index-covered
  • Migration DelayedQueueJDBC with initial HSQL Support #2 added to upgrade existing databases

Affected Adapters
H2, HSQLDB, SQLite, PostgreSQL, MariaDB, MS SQL Server, Oracle

Example

-- Before: non-deterministic when scheduledAt values collide
SELECT TOP 5 id FROM queue 
WHERE pKind = ? AND scheduledAt <= ?
ORDER BY scheduledAt

-- After: deterministic, index-covered
SELECT TOP 5 id FROM queue 
WHERE pKind = ? AND scheduledAt <= ?
ORDER BY scheduledAt, id

Index (pKind, scheduledAt, id) satisfies WHERE clause filtering and complete ORDER BY without additional sorting.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • settings.data.microsoft.com
    • Triggering command: /usr/bin/dotnet dotnet /app/sqlceip/EdgeTelemetry.dll 00000001-0000-002a-0000-080000000100 funfix/delayedqueue/jvm/internals/jdbc/mariadb/ vm/s�� vm/src/main/kotlin/org/funfix/delayedqueue/jvm/DelayedQueue.kt vm/src/main/kotlin/org/funfix/delayedqueue/jvm/DelayedQueueInMemory.kt sBatchSizeLimit().html (dns block)
  • vortex.data.microsoft.com
    • Triggering command: /usr/bin/dotnet dotnet /app/sqlceip/EdgeTelemetry.dll 00000001-0000-002a-0000-080000000100 funfix/delayedqueue/jvm/internals/jdbc/mariadb/ vm/s�� vm/src/main/kotlin/org/funfix/delayedqueue/jvm/DelayedQueue.kt vm/src/main/kotlin/org/funfix/delayedqueue/jvm/DelayedQueueInMemory.kt sBatchSizeLimit().html (dns block)
  • www.example.com
    • Triggering command: /opt/oracle/product/23ai/dbhomeFree/bin/oracle oracleFREE (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) docker0 -p tcp -d 172.17.0.4 cb8649b3e444b810554843cca28f75dd65a/log.json 1433 -j ACCEPT bash --no�� cb8649b3e444b810554843cca28f75dd65a (dns block)
    • Triggering command: /opt/oracle/product/23ai/dbhomeFree/bin/oracle oracleFREE (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) 3a86a0ed3c4d5acc32a1e65cc5372f841ca87fe7fe6ddbd03417ea16abb85772 9 -c 3f22f63db41c6b5b-w (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This is probably a non-deterministic test failure, try and fix it.

> Task :delayedqueue-jvm:test

DelayedQueueJDBCH2Test > tryPollMany_respectsBatchSizeLimit() FAILED
    org.opentest4j.AssertionFailedError at DelayedQueueJDBCContractTestBase.java:216

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits February 7, 2026 20:53
Add secondary sort by id column to ORDER BY clauses in both
acquireManyOptimistically and selectFirstAvailableWithLock methods
across all database adapters (H2, HSQLDB, SQLite, PostgreSQL,
MariaDB, MS SQL Server, Oracle).

This ensures deterministic ordering when multiple messages have
the same scheduledAt timestamp, which became a problem after
H2 upgrade from 2.3.232 to 2.4.240.

Co-authored-by: alexandru <11753+alexandru@users.noreply.github.com>
Add 'id' column to KindPlusScheduledAtIndex in all database migrations
to ensure ORDER BY (scheduledAt, id) queries are fully index-covered.

Also add migration #2 to update existing databases by dropping and
recreating the index with the additional column.

This ensures both deterministic ordering and optimal query performance
without requiring additional sort operations.

Co-authored-by: alexandru <11753+alexandru@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix non-deterministic test failure in DelayedQueueJDBCH2Test Fix non-deterministic ordering in message polling queries Feb 7, 2026
Copilot AI requested a review from alexandru February 7, 2026 21:09
Migration(
sql =
"""
DROP INDEX IF EXISTS "${tableName}__KindPlusScheduledAtIndex";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot This isn't deployed in any production systems yet, I don't want a second migration.

@alexandru alexandru closed this Feb 7, 2026
@alexandru alexandru deleted the copilot/fix-non-deterministic-test-failure branch February 7, 2026 21:15
Copilot AI requested a review from alexandru February 7, 2026 21:15
Copilot stopped work on behalf of alexandru due to an error February 7, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants