Merged
Conversation
Add two-phase commit support for distributed transactions. Participant: PreparedTransactions registry, HTTP endpoints for prepare/commit/abort, prepare_reducer() on ModuleHost. Coordinator: call_reducer_on_db_2pc host function (ABI spacetime_10.5), post-commit sends /2pc/commit to participants, on failure sends abort. Bindings: FFI and safe wrapper for call_reducer_on_db_2pc. Smoketests: cross_db_2pc with happy path and abort path.
The persistence barrier prevents speculative transactions from being persisted to the durability worker while a 2PC PREPARE is pending. When prepare_reducer commits a transaction: 1. The PREPARE is sent to the durability worker normally. 2. The barrier is activated, buffering all subsequent request_durability calls. 3. prepare_reducer waits for the PREPARE offset to become durable. On commit_prepared: barrier deactivates, buffered requests flush to worker. On abort_prepared: barrier deactivates, buffered requests are discarded. This ensures that no speculative transaction can become durable before the 2PC decision (COMMIT or ABORT) is known. Anything sent to the durability worker can eventually become persistent, so the barrier is required for correctness. RelationalDB.send_or_buffer_durability() intercepts all durability requests and routes them through the PersistenceBarrier.try_buffer() check.
The persistence barrier is a database-layer concern (it intercepts durability requests), not a host-layer concern. Move it out of prepared_tx.rs into relational_db.rs where it belongs. prepared_tx.rs now only contains PreparedTxInfo and PreparedTransactions (the host-layer registry for tracking in-flight 2PC transactions).
The barrier should block persistence only until the PREPARE record is confirmed durable. Once durable, subsequent transactions can persist normally. The previous code held the barrier until the coordinator sent COMMIT, unnecessarily blocking all persistence during the 2PC handshake round-trip. Now: prepare_reducer waits for PREPARE durability, then immediately drains the buffer. commit_prepared just removes from the registry. abort_prepared still needs to invert in-memory state (TODO).
Documents the full pipelined 2PC protocol for coordinator and participant, including the persistence barrier, serializable isolation (participant holds MutTxId across all calls in a coordinator transaction), two-phase participant response (immediate result + deferred PREPARED after durability), abort paths, commitlog format, and replay semantics. Identifies the open problem: MutTxId is !Send but must be held across multiple HTTP requests on the participant side.
Replace the open problem section with the concrete solution: a dedicated blocking thread per participant transaction holds the MutTxId for its entire lifetime. Async HTTP handlers communicate via channels. The MutTxId never crosses a thread boundary. Includes the TxCommand enum design, session management, and ASCII diagram of the HTTP handler / blocking thread interaction.
Instead of inventing a new threading model, reuse the same std::thread::scope + blocking_recv pattern that call_reducer_on_db already uses. The participant's thread executes the reducer, sends the result, then blocks on a channel waiting for the next command. The MutTxId stays alive on that same thread. Includes updated ASCII diagram showing the coordinator/participant thread interaction, the session-based HTTP protocol, and how the persistence barrier arms before commit.
Remove the Armed state. No race is possible because the barrier is activated on the same thread that just released the write lock, and the PREPARE is sent to the durability worker directly (not through send_or_buffer_durability) before the barrier activates.
Shubham8287
changed the base branch from
phoebe/tpcc-distributed-naive-http-requests
to
phoebe/tpcc/reducer-return-value
aasoni
changed the base branch from
phoebe/tpcc/reducer-return-value
to
jdetter/tpcc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
API and ABI breaking changes
Expected complexity level and risk
Testing