Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/adr/ADR-003-integer-cents.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,7 @@ The only special case is royalty splits where division produces remainders (e.g.
- All monetary values are integers in the codebase — no decimal types anywhere
- API accepts and returns cent values (documented in API contract)
- Division for royalty splits requires explicit rounding strategy (see ADR-005)

## Addendum (2026-04-12)

Centi-cents (1/100 of a cent, stored as `wallets.fractional_balance`) extend this principle for sub-cent royalty accrual. See ADR-010.
4 changes: 3 additions & 1 deletion docs/adr/ADR-005-platform-royalty-remainder.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# ADR-005: Platform Receives Royalty Remainder

**Status:** Accepted
**Status:** Superseded by ADR-010
**Date:** 2026-02-26

> **Superseded:** This ADR describes the original floor-only rounding strategy. It has been replaced by ADR-010 (centi-cent accrual), which eliminates the systematic under-payment of authors. The platform still receives the remainder after sweep, but over time the amounts converge to the nominal 30%.

## Context

Royalty splits (70% author, 30% platform) applied to integer cent values can produce remainders. For example: 70% of 99 cents = 69.3 cents — not a valid integer. A rounding strategy must be defined.
Expand Down
32 changes: 32 additions & 0 deletions docs/adr/ADR-008-running-totals-for-reporting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# ADR-008: Running Totals for Reporting

**Status:** Accepted
**Date:** 2026-04-12

## Context

`ReportsProcessor.aggregate()` ran a `GROUP BY` over the entire `ledger` table inside a `REPEATABLE READ` transaction. At 100M+ rows this causes a full table scan, consumes memory proportional to result size, and holds a long-running transaction on the operational database.

## Decision

A new `ledger_totals` table maintains pre-aggregated running sums per ledger type:

```sql
type ledger_type PRIMARY KEY
total BIGINT NOT NULL DEFAULT 0
```

The table is updated inside the same transaction as every ledger insert using `INSERT ... ON CONFLICT DO UPDATE`. `ReportsProcessor` replaces the `GROUP BY` with a single `SELECT * FROM ledger_totals` (4 rows, no scan).

## Reasoning

- The running total is always consistent with the ledger because it is updated in the same transaction. No eventual consistency.
- O(1) read path replaces O(N) scan.
- `BIGINT` is chosen because the sum of 100M+ cent-denominated values can exceed INT4's ~2.1 billion max.
- Upsert (`INSERT ... ON CONFLICT DO UPDATE`) is used rather than bare `UPDATE` to handle fresh databases where the row may not yet exist.

## Consequences

- Reporting is O(1) regardless of ledger size.
- Write path has a small additional upsert per transaction (negligible vs. ledger insert cost).
- Migration includes a backfill from existing ledger data.
34 changes: 34 additions & 0 deletions docs/adr/ADR-009-redis-idempotency-cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# ADR-009: Redis Idempotency Cache

**Status:** Accepted
**Date:** 2026-04-12

## Context

Every purchase request hit Postgres with a `SELECT WHERE idempotency_key = ?` before the transaction. Under high retry volume this is unnecessary latency and DB load for requests that already have a known result.

## Decision

Redis SETNX edge cache. Before the DB check:

```
SET idempotency:<key> 'processing' EX 86400 NX
```

- If NX returns `null` (key exists): read the cached value and return or `409`.
- If NX returns `'OK'` (new request): proceed to DB.
- Cache the result after commit.
- Delete sentinel on transaction failure so clients can retry.

## Reasoning

- Redis is a latency shortcut — the DB remains the correctness backstop. The `UNIQUE` constraint on `purchases.idempotency_key` and the existing DB SELECT are untouched.
- Any request that slips past Redis (eviction, cold start, Redis unavailable) falls through to the DB path.
- The `'processing'` sentinel must be deleted on transaction failure — otherwise recoverable failures (e.g., 402 insufficient funds) would block client retries for 24 hours.
- TTL of 86400s (24 hours) covers any reasonable client retry window.

## Consequences

- Repeated requests with a completed purchase are served from Redis without hitting Postgres.
- Redis unavailability degrades gracefully (falls through to DB).
- A separate `RedisModule` is required because BullMQ does not expose its internal ioredis instance via NestJS DI.
40 changes: 40 additions & 0 deletions docs/adr/ADR-010-centi-cent-fractional-accrual.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# ADR-010: Centi-cent Fractional Accrual

**Status:** Accepted
**Date:** 2026-04-12
**Supersedes:** ADR-005

## Context

`Math.floor(price * 70 / 100)` always floors the author royalty, giving every sub-cent remainder to the platform. Over millions of microtransactions this is a systematic wealth transfer away from authors.

Example over three 99-cent transactions: author should receive 3 × 69.3 = 207.9 cents but actually receives 3 × 69 = 207 cents; platform over-receives ~1 cent per 3 transactions.

## Decision

Per-author centi-cent accrual. Track each author wallet's fractional royalty remainder at centi-cent precision (1/100 of a cent, stored as integer 0–99 in `wallets.fractional_balance`). When accumulated centi-cents reach 100, sweep one whole cent into the author's balance within the same transaction.

```typescript
const exactNumerator = itemPrice * AUTHOR_ROYALTY_PERCENT; // e.g. 99*70 = 6930
const authorFloorCents = Math.floor(exactNumerator / 100); // 69
const remainderCentiCents = exactNumerator % 100; // 30
const newFractional = authorWallet.fractionalBalance + remainderCentiCents;
const sweepCents = Math.floor(newFractional / 100); // whole cents to sweep
const leftoverCenti = newFractional % 100; // stored back
const totalAuthorCents = authorFloorCents + sweepCents; // actual credit
const platformCut = itemPrice - totalAuthorCents; // always = itemPrice
```

## Reasoning

- Money conservation holds — `totalAuthorCents + platformCut === itemPrice` always.
- Stays within ADR-003's integer-only philosophy — centi-cents are a finer-grained integer unit.
- The `SELECT FOR UPDATE` on the author wallet row means `fractionalBalance` can be set directly (no SQL arithmetic needed — we hold the lock).
- No background job required.

## Consequences

- Author wallets gain a `fractional_balance` column (0–99).
- Authors receive the correct economic share over time rather than systematically losing sub-cent remainders.
- Platform receives slightly less on average (closer to the nominal 30%).
- Ledger entries record `totalAuthorCents` and `platformCut` (actual money moved, not nominal percentages).
32 changes: 26 additions & 6 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ Wallet is a digital wallet API that manages integer-cent balances, purchase tran
graph LR
Client -->|HTTP| API[NestJS API :3000]
API -->|Drizzle ORM| PG[(PostgreSQL 16)]
API -->|BullMQ| Redis[(Redis 7)]
API -->|ioredis - idempotency cache| Redis[(Redis 7)]
API -->|BullMQ| Redis
Redis -->|Job| Worker[Report Processor]
Worker -->|Drizzle ORM| PG
```
Expand Down Expand Up @@ -84,6 +85,13 @@ erDiagram
jsonb result
timestamp completed_at
}

ledger_totals {
enum type PK "ledger_type"
bigint total "running sum"
}

ledger ||--o{ ledger_totals : "aggregated into"
```

## Concurrency & Transactions
Expand All @@ -97,17 +105,26 @@ Each deposit runs inside a transaction that locks the wallet row with `SELECT FO
A purchase involves three wallets: buyer, author, and platform. All three are locked in a single query ordered by `id ASC` (`FOR UPDATE`) to enforce consistent lock acquisition and prevent deadlocks. Within the same transaction:

1. Buyer balance is decremented by the item price
2. Author receives `floor(price * 70 / 100)` (author royalty)
3. Platform receives the remainder (`price - authorCut`)
2. Author receives floor royalty plus any accrued centi-cents that crossed 100 (see ADR-010)
3. Platform receives the remainder (`price - totalAuthorCents`)
4. A purchase record and three ledger entries are inserted

If PostgreSQL detects a deadlock (`40P01`), the service catches it and returns `409 Conflict` with a retry hint.

### Idempotency

Each purchase carries a client-owned `Idempotency-Key` header (UUID). Before entering the transaction, the service checks for an existing purchase with that key:
Each purchase carries a client-owned `Idempotency-Key` header (UUID). Before entering the transaction, the service checks a Redis edge cache then the database:

**Redis SETNX (fast path):** `SET idempotency:<key> 'processing' EX 86400 NX`

- NX returns `null` (key exists): read cached value — return completed purchase or `409`
- NX returns `'OK'` (new request): proceed to DB check below
- Redis unavailable: log warning, fall through to DB (degraded but not broken)
- Transaction failure: sentinel is deleted so clients can retry

**DB check (cold-start / Redis miss):**

- **Completed + same payload** — returns the cached result (safe replay)
- **Completed + same payload** — populate Redis, return cached result (safe replay)
- **Completed + different payload** — `409 Conflict` (payload drift)
- **Still in flight** — `409 Conflict`

Expand All @@ -118,7 +135,7 @@ The `idempotency_key` column has a unique constraint — concurrent inserts with
Financial reports are generated asynchronously:

1. `POST /reports/financial` — inserts a report row with status `queued`, enqueues a BullMQ job, returns `{ jobId, status }`
2. **ReportsProcessor** (BullMQ worker) picks up the job, sets status to `processing`, runs an aggregation query inside a `REPEATABLE READ` transaction, and stores the JSONB result
2. **ReportsProcessor** (BullMQ worker) picks up the job, sets status to `processing`, reads pre-aggregated totals from `ledger_totals` (O(1), no table scan), and stores the JSONB result
3. `GET /reports/financial/:jobId` — polls the report status and result, scoped to the requesting user

If Redis is down when enqueuing, the report is immediately marked `failed` rather than left orphaned in `queued`.
Expand All @@ -134,6 +151,9 @@ Architectural decisions are recorded as ADRs:
- [ADR-005: Platform Receives Royalty Remainder](adr/ADR-005-platform-royalty-remainder.md)
- [ADR-006: BullMQ for Async Report Generation](adr/ADR-006-bullmq-async-reports.md)
- [ADR-007: Idempotency Key Owned by Client](adr/ADR-007-client-owned-idempotency-key.md)
- [ADR-008: Running Totals for Reporting](adr/ADR-008-running-totals-for-reporting.md)
- [ADR-009: Redis Idempotency Cache](adr/ADR-009-redis-idempotency-cache.md)
- [ADR-010: Centi-cent Fractional Accrual](adr/ADR-010-centi-cent-fractional-accrual.md)

## Security

Expand Down
13 changes: 13 additions & 0 deletions drizzle/0001_modern_rogue.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
CREATE TABLE "ledger_totals" (
"type" "ledger_type" PRIMARY KEY NOT NULL,
"total" bigint DEFAULT 0 NOT NULL
);
--> statement-breakpoint
ALTER TABLE "wallets" ADD COLUMN "fractional_balance" integer DEFAULT 0 NOT NULL;--> statement-breakpoint
ALTER TABLE "wallets" ADD CONSTRAINT "wallets_fractional_balance_non_negative" CHECK ("wallets"."fractional_balance" >= 0);--> statement-breakpoint
ALTER TABLE "wallets" ADD CONSTRAINT "wallets_fractional_balance_lt_100" CHECK ("wallets"."fractional_balance" < 100);

-- Backfill running totals from existing ledger data
INSERT INTO ledger_totals (type, total)
SELECT type, COALESCE(SUM(amount), 0) FROM ledger GROUP BY type
ON CONFLICT (type) DO NOTHING;
Comment thread
idris-builds marked this conversation as resolved.
Loading
Loading