Add first-pass TPC-C module and rust benchmark runner#4711
Draft
joshua-spacetime wants to merge 3 commits intomasterfrom
Draft
Add first-pass TPC-C module and rust benchmark runner#4711joshua-spacetime wants to merge 3 commits intomasterfrom
joshua-spacetime wants to merge 3 commits intomasterfrom
Conversation
e57da6c to
b05b41a
Compare
b05b41a to
d84b7a7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds a first-pass end-to-end TPC-C implementation for SpacetimeDB, split into a Rust module and a Rust benchmark harness.
The change includes:
modules/tpccmodule that models the TPC-C schema, integrity checks, loader reducers, terminal-facing transaction entrypoints, and deferredDeliveryprocessingtools/tpcc-runnerbinary that can load data, run a local driver, or coordinate multiple remote driversWhat The Module Implements
The module adds the core TPC-C tables:
warehousedistrictcustomerhistoryitemstockoordernew_order_roworder_linedelivery_jobdelivery_completionThe module defines secondary access-path indexes needed for TPC-C behavior, including last-name lookups, recent-order lookups, stock lookups, and delivery scans.
Money is represented as integer cents and tax/discount are represented in basis points, so the implementation stays in integer math.
The module exposes:
reset_tpccnew_order,payment,order_status,stock_level, andqueue_deliverydelivery_progressandfetch_delivery_completionsrun_delivery_jobthat performs deferred delivery workA notable nuance is that the terminal-facing write transactions are implemented as procedures too, not just the read-only ones. That is deliberate: the runner needs direct typed return values for terminal-visible results and acknowledgements, so procedures are used for the interactive path while reducers remain the bulk-load and scheduled-work path.
Manual Logical Primary Key Enforcement
Because SpacetimeDB does not support composite primary keys, the module now enforces TPC-C logical primary-key uniqueness explicitly.
Reusable helper functions handle inserts into the no-PK composite-key tables by:
That logic is used consistently in:
New-OrderThis keeps the TPC-C uniqueness semantics while avoiding extra stored key columns.
A direct consequence is that mutable rows on those tables are updated by replacement rather than by native PK update:
That pattern is used for frequently updated TPC-C rows like:
district.d_next_o_idcustomerbalances/payment countersstockquantity/YTD/order countersoorder.o_carrier_idorder_line.ol_delivery_dTransaction Semantics
New-Order:d_next_o_idoorder,new_order_row, andorder_linePayment:c_idor by last namec_datafor bad-credit customers and truncates it to the TPC-C maxhistoryOrder-Status:Stock-Level:d_next_o_idBTreeSetDelivery:queue_deliveryvalidates inputs and inserts adelivery_jobscheduled_idand timestampdelivery_completionrow when district 10 is doneThat one-district-per-invocation design is intentional. It keeps background delivery work incremental and matches the TPC-C allowance for delivery to complete across multiple database transactions.
Integrity And Validation
Referential integrity is enforced in module logic rather than declarative foreign keys.
The module checks:
Because the transaction logic runs inside a single transactional context, any validation failure aborts the transaction and rolls back the whole operation.
Loader
The loader builds the initial TPC-C dataset in Rust and sends it in reducer batches.
It supports optional
reset_tpccfirst, then loads:itemrows(warehouse, item)pairImportant nuances in the loader:
BCvsGCcustomers andORIGINALdata injectiond_next_o_id = 30011..2100are pre-delivered and orders2101..3000start as new ordersRunner Architecture
tpcc-runnerhas three subcommands:loaddrivercoordinatorThe runner is designed to work both locally and remotely.
A local run uses one driver process and no coordinator.
A distributed run uses one or more remote driver processes plus an optional coordinator.
The driver model is:
(warehouse_id, district_id)assignment per terminalThe current terminal assignment model is spec-shaped:
DISTRICTS_PER_WAREHOUSE = 10(warehouse, district)warehouse_count * 10The transaction mix generator uses the standard TPC-C ratios:
New-Order45%Payment43%Order-Status4%Delivery4%Stock-Level4%It also applies the expected input distributions:
New-Orderhas 5-15 order linesNew-Orderuses a 1% invalid final item for rollback behaviorNew-Ordermakes 1% of order lines remote when multiple warehouses existPaymentis remote 15% of the time when multiple warehouses existPaymentandOrder-Statusselect by last name 60% of the timeThe SDK wrapper intentionally turns callback-style reducer/procedure APIs into blocking request/response helpers using
sync_channelplus a timeout. That keeps the loader and terminal loops simple and deterministic.Distributed Coordination
The coordinator is a small HTTP service built with
axum.It provides:
POST /registerfor driver registrationGET /schedulefor drivers to poll until a shared schedule is availablePOST /summaryfor per-driver summary submissionOnce all expected drivers register, the coordinator publishes a common warmup/measurement schedule.
Drivers still write their own local outputs.
The coordinator aggregates submitted summaries and writes an aggregate
summary.json.Raw
txn_events.ndjsonfiles remain local to each driver process.Reporting
Each driver writes:
summary.jsontxn_events.ndjsonThe event log records one measured transaction per line with:
The driver summary includes:
tpmc_like, computed from successfulNew-Orderthroughput over the measurement windowDeliveryis measured in two different ways:queue_deliveryrequest is counted as the terminal transactiondelivery_completionrows harvested after the runOperational Defaults
Notable defaults:
true10 * warehousesterminalstpcc-results/<run-id>/<driver-id>/infofor the runnerConfiguration can come from CLI flags or a TOML config file, with CLI taking precedence.
Nuances And Deliberate Non-Goals
This is a serious first pass, but it is not an official audited TPC-C kit.
Deliberate boundaries:
tpmc_like, not an official publishedtpmCresult packageunstablefeature