Skip to content

Refine live subscriptions & materialization#250

Merged
jamals86 merged 1 commit into
mainfrom
029-datafusion-modernization
May 4, 2026
Merged

Refine live subscriptions & materialization#250
jamals86 merged 1 commit into
mainfrom
029-datafusion-modernization

Conversation

@jamals86
Copy link
Copy Markdown
Member

@jamals86 jamals86 commented May 4, 2026

Add validation and helper functions for websocket subscription options (prevent last_rows > batch_size) and return early on invalid options. Refactor initial data fetching: introduce TableCapabilities, reduce repeated schema lookups, move heavy row materialization to a blocking task when large, and centralize materialization logic with many unit/integration tests covering resume/last-row behaviors. Simplify connection metadata by removing a custom string-intern pool in favor of Arc-owned strings and update tests accordingly. Optimize notification dispatch for single and small fan-outs (add dispatch_one) to avoid extra cloning/spawning. Update benchmark tooling: add --output-dir, reorganize chat realtime benchmark into chat_runtime, and adjust benchmark registration/selection and scripts. Minor formatting and dependency lock updates.

Add validation and helper functions for websocket subscription options (prevent last_rows > batch_size) and return early on invalid options. Refactor initial data fetching: introduce TableCapabilities, reduce repeated schema lookups, move heavy row materialization to a blocking task when large, and centralize materialization logic with many unit/integration tests covering resume/last-row behaviors. Simplify connection metadata by removing a custom string-intern pool in favor of Arc-owned strings and update tests accordingly. Optimize notification dispatch for single and small fan-outs (add dispatch_one) to avoid extra cloning/spawning. Update benchmark tooling: add --output-dir, reorganize chat realtime benchmark into chat_runtime, and adjust benchmark registration/selection and scripts. Minor formatting and dependency lock updates.
@jamals86 jamals86 merged commit 6524c76 into main May 4, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant