Skip to content

Conversation

@samwillis
Copy link
Collaborator

combines #960 and #962 in attempt to see if latter fixes the CI issue...

samwillis and others added 11 commits December 3, 2025 09:10
Replace hash-based object ID tie-breaking with direct key comparison
for deterministic ordering when ORDER BY values are equal.

- Use row key directly as tie-breaker (always string | number, unique per row)
- Remove globalObjectIdGenerator dependency
- Simplify TaggedValue from [K, V, Tag] to [K, T] tuple
- Clean up helper functions (tagValue, getKey, getVal, getTag)

This ensures stable, deterministic ordering across page reloads and
eliminates potential hash collisions.
…exes

- SortedMap: add key-based tie-breaking for deterministic ordering
- SortedMap: optimize to skip value comparison when no comparator provided
- BTreeIndex: sort keys within same indexed value for deterministic order
- BTreeIndex: add fast paths for empty/single-key sets
- CollectionStateManager: always use SortedMap for deterministic iteration
- Extract compareKeys utility to utils/comparison.ts
- Add comprehensive tests for deterministic ordering behavior
push down multiple orderby predicates to load subset

split order by cursor predicate build into two, inprecise wider band for local lading, precise for the sync loadSubset

new e2e tests for composite orderby and pagination

changeset

when doing gt/lt comparisons to a bool cast to string

fix: use non-boolean columns in multi-column orderBy e2e tests

Electric/PostgreSQL doesn't support comparison operators (<, >, <=, >=)
on boolean types. Changed tests to use age (number) and name (string)
columns instead of isActive (boolean) to avoid this limitation.

The core multi-column orderBy functionality still works correctly -
this is just a test adjustment to work within Electric's SQL parser
constraints.
…e in loadSubset

- Add CursorExpressions type with whereFrom, whereCurrent, and lastKey
- LoadSubsetOptions.where no longer includes cursor - passed separately via cursor property
- Add offset to LoadSubsetOptions for offset-based pagination support
- Electric sync layer makes two parallel requestSnapshot calls when cursor present
- Query collection serialization includes offset for query key generation

This allows sync layers to choose between cursor-based or offset-based pagination,
and Electric can efficiently handle tie-breaking with targeted requests.
…r expressions

The test mock's loadSubset handler now handles the new cursor property
in LoadSubsetOptions by combining whereCurrent (ties) and whereFrom (next page)
data, deduplicating by id, and re-sorting.
Changed parallel requestSnapshot calls to sequential to avoid potential
issues with concurrent snapshot requests that may cause timeouts in CI.
Instead of making two separate requestSnapshot calls (one for whereFrom,
one for whereCurrent), combine them using OR into a single request.
This avoids potential issues with multiple sequential snapshot requests
that were causing timeouts in CI.

The combined expression (whereFrom OR whereCurrent) matches the original
behavior where cursor was combined with the where clause.
…sync

For live queries using on-demand sync mode, the collection was being
marked as ready immediately when source collections were ready, even
though the initial loadSubset hadn't completed yet. This meant
`isLoading` was false while data was still being fetched.

This fix ensures that:
1. Live queries with on-demand sources track the initial loadSubset
   promise and show `isLoading: true` until it completes
2. The collection status remains `loading` until the first data load
   finishes
3. Subsequent loadSubset calls (pagination/windowing) do NOT affect
   the ready status - only the first load matters

Changes:
- Add `hasMarkedReady` and `hasSetupLoadingListener` flags to track
  initial ready state in CollectionConfigBuilder
- Modify `updateLiveQueryStatus()` to wait for first loadSubset to
  complete before calling `markReady()`
- Update `subscribeToMatchingChanges()` in CollectionSubscriber to
  track the initial loadSubset promise for on-demand sources
- Add comprehensive tests for the new behavior
Update test to wait for the initial loadSubset promise to resolve
before checking the live query status. This is needed after the fix
that makes live queries show loading status during initial on-demand
data fetch.
…tus-01TNsEPERmfs3ub3e9iDRAzK' into samwillis/cursor-pagination-BACKUP
@changeset-bot
Copy link

changeset-bot bot commented Dec 3, 2025

🦋 Changeset detected

Latest commit: fde044e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 14 packages
Name Type
@tanstack/db Patch
@tanstack/electric-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/db-ivm Patch
@tanstack/angular-db Patch
@tanstack/db-collection-e2e Patch
@tanstack/offline-transactions Patch
@tanstack/powersync-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Dec 3, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@963

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@963

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@963

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@963

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@963

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@963

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@963

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@963

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@963

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@963

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@963

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@963

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@963

commit: fde044e

@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

Size Change: +1.08 kB (+1.24%)

Total Size: 88.2 kB

Filename Size Change
./packages/db/dist/esm/collection/state.js 3.42 kB -8 B (-0.23%)
./packages/db/dist/esm/collection/subscription.js 3.03 kB +481 B (+18.89%) ⚠️
./packages/db/dist/esm/indexes/btree-index.js 1.96 kB +85 B (+4.54%) 🔍
./packages/db/dist/esm/query/compiler/order-by.js 1.46 kB +206 B (+16.48%) ⚠️
./packages/db/dist/esm/query/live/collection-config-builder.js 5.43 kB +102 B (+1.92%)
./packages/db/dist/esm/query/live/collection-subscriber.js 1.79 kB +57 B (+3.28%)
./packages/db/dist/esm/SortedMap.js 1.3 kB +127 B (+10.81%) ⚠️
./packages/db/dist/esm/utils/comparison.js 885 B +33 B (+3.87%)
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.39 kB
./packages/db/dist/esm/collection/changes.js 977 B
./packages/db/dist/esm/collection/events.js 388 B
./packages/db/dist/esm/collection/index.js 3.24 kB
./packages/db/dist/esm/collection/indexes.js 1.1 kB
./packages/db/dist/esm/collection/lifecycle.js 1.67 kB
./packages/db/dist/esm/collection/mutations.js 2.31 kB
./packages/db/dist/esm/collection/sync.js 2.37 kB
./packages/db/dist/esm/deferred.js 207 B
./packages/db/dist/esm/errors.js 4.19 kB
./packages/db/dist/esm/event-emitter.js 748 B
./packages/db/dist/esm/index.js 2.64 kB
./packages/db/dist/esm/indexes/auto-index.js 742 B
./packages/db/dist/esm/indexes/base-index.js 766 B
./packages/db/dist/esm/indexes/lazy-index.js 1.1 kB
./packages/db/dist/esm/indexes/reverse-index.js 513 B
./packages/db/dist/esm/local-only.js 837 B
./packages/db/dist/esm/local-storage.js 2.1 kB
./packages/db/dist/esm/optimistic-action.js 359 B
./packages/db/dist/esm/paced-mutations.js 496 B
./packages/db/dist/esm/proxy.js 3.75 kB
./packages/db/dist/esm/query/builder/functions.js 733 B
./packages/db/dist/esm/query/builder/index.js 3.96 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 917 B
./packages/db/dist/esm/query/compiler/evaluators.js 1.35 kB
./packages/db/dist/esm/query/compiler/expressions.js 430 B
./packages/db/dist/esm/query/compiler/group-by.js 1.8 kB
./packages/db/dist/esm/query/compiler/index.js 1.96 kB
./packages/db/dist/esm/query/compiler/joins.js 2 kB
./packages/db/dist/esm/query/compiler/select.js 1.07 kB
./packages/db/dist/esm/query/expression-helpers.js 1.43 kB
./packages/db/dist/esm/query/ir.js 673 B
./packages/db/dist/esm/query/live-query-collection.js 360 B
./packages/db/dist/esm/query/live/collection-registry.js 264 B
./packages/db/dist/esm/query/live/internal.js 130 B
./packages/db/dist/esm/query/optimizer.js 2.56 kB
./packages/db/dist/esm/query/predicate-utils.js 2.91 kB
./packages/db/dist/esm/query/subset-dedupe.js 921 B
./packages/db/dist/esm/scheduler.js 1.3 kB
./packages/db/dist/esm/strategies/debounceStrategy.js 247 B
./packages/db/dist/esm/strategies/queueStrategy.js 428 B
./packages/db/dist/esm/strategies/throttleStrategy.js 246 B
./packages/db/dist/esm/transactions.js 2.9 kB
./packages/db/dist/esm/utils.js 881 B
./packages/db/dist/esm/utils/browser-polyfills.js 304 B
./packages/db/dist/esm/utils/btree.js 5.61 kB
./packages/db/dist/esm/utils/index-optimization.js 1.51 kB
./packages/db/dist/esm/utils/type-guards.js 157 B

compressed-size-action::db-package-size

@samwillis samwillis closed this Dec 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

Size Change: 0 B

Total Size: 3.35 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 225 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.17 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.12 kB
./packages/react-db/dist/esm/useLiveSuspenseQuery.js 431 B
./packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants