Skip to content

Conversation

@joshua-spacetime
Copy link
Collaborator

@joshua-spacetime joshua-spacetime commented Dec 16, 2025

Description of Changes

With the addition of module-defined views, subscriptions are no longer read-only as they may invoke view materialization.

The way this works is that a subscription starts off as a mutable transaction, materializes views if necessary, and then downgrades to a read-only transaction to evaluate the subscription.

Before this patch, we were calling commit_downgrade directly on the MutTxId in order to downgrade the transaction. This would update the in-memory CommittedState, but it wouldn't make the transaction durable.

This would result in us incrementing the transaction offset of the in-memory CommittedState without writing anything to the commitlog. This in turn would invalidate snapshots as they would be pointing further ahead into the commitlog than they should, and so when replaying from a snapshot we would potentially skip over commits that were not included in the snapshot.

This patch changes those call sites to use RelationalDB::commit_tx_downgrade which both updates the in-memory state and makes the transaction durable.

NOTE: The fact that views are materialized is purely an implementation detail at this point in time. And technically view tables are ephemeral meaning they are not persisted to the commitlog. So the real bug here was that we were updating the tx offset of the in-memory committed state at all. This is technically fixed by #3884 and so after #3884 lands this change becomes a no-op. However, we still shouldn't be calling commit and commit_downgrade directly on a MutTxId since in most cases it is wrong to bypass the durability layer. And without this change, the bug would still be present were view tables not ephemeral, which they may not be at some point in the future.

API and ABI breaking changes

None

Expected complexity level and risk

  1. The change itself is trivial, the bug is not.

Testing

Adding an automated test for this is not so straightforward. First it's view related which means we don't have many options apart from a smoke test, but I don't believe the smoke tests have a mechanism for replaying the commitlog.

If transaction offsets are supposed to be linear, without any gaps, then it would be useful to assert that on each append, in which case we could write a smoke test that would fail as soon as the offsets diverged.

@joshua-spacetime joshua-spacetime requested a review from kim December 16, 2025 23:26
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/make-subscribe-durable branch from e7538eb to 6e2999d Compare December 17, 2025 00:34
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/make-subscribe-durable branch from 6e2999d to a140e31 Compare December 17, 2025 06:35
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/make-subscribe-durable branch from a140e31 to 5e09d56 Compare December 17, 2025 16:36
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/make-subscribe-durable branch from 5e09d56 to 6149298 Compare December 17, 2025 22:15
@joshua-spacetime joshua-spacetime added this pull request to the merge queue Dec 18, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 18, 2025
@joshua-spacetime joshua-spacetime added this pull request to the merge queue Dec 18, 2025
Merged via the queue into master with commit cb5e711 Dec 18, 2025
40 of 42 checks passed
@joshua-spacetime joshua-spacetime deleted the joshua/fix/make-subscribe-durable branch December 18, 2025 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants