Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
e89dd7d
Rebase on upstream hourly, add AI/LLM PR review
gburd Mar 10, 2026
1eef97d
dev setup v34
gburd Apr 29, 2026
969f360
tepid dev additions
gburd May 11, 2026
10fe0b3
Add tests to cover a variety of heap HOT update behaviors
gburd Mar 10, 2026
f56067e
Identify modified indexed attributes in the executor on UPDATE
gburd Apr 1, 2026
b53e747
Replace TU_UpdateIndexes with per-index bitmapset tracking
gburd Apr 1, 2026
38cfe72
Add on-disk layout for the HOT-indexed tombstone item
gburd May 6, 2026
f208781
Add RelationGetIndexedAttrs() per-index attribute bitmap accessor
gburd May 6, 2026
a92b712
Use RelationGetIndexedAttrs to decide ii_IndexUnchanged precisely
gburd May 7, 2026
c6f5346
Add heap_build_hot_indexed_tombstone builder for HOT-indexed tombstones
gburd May 7, 2026
4ebdf57
Introduce HeapUpdateHotMode tri-state (preparation for HOT-indexed)
gburd May 7, 2026
228552e
Teach pruneheap/vacuumlazy to skip HOT-indexed tombstones
gburd May 7, 2026
b214dc7
Implement the HOT-indexed write path in heap_update
gburd May 7, 2026
7b36ef4
Add HOT-indexed reader path: chain-walk recheck + direct heap-only lo…
gburd May 7, 2026
203b171
Reclaim orphaned HOT-indexed tombstones during prune
gburd May 7, 2026
6e3d868
Reader: fixes for CLUSTER, unique checks, and stale dedup
gburd May 7, 2026
2aafda2
BitmapHeapScan: force recheck on HOT-indexed chain walks
gburd May 7, 2026
88801cf
Reader: close remaining HOT-indexed correctness gaps in regression
gburd May 8, 2026
f27ff3e
Enable HOT-indexed updates by default
gburd May 8, 2026
9c98d62
heap_update: harden HOT-indexed fit check; filter stale catalog scan …
gburd May 8, 2026
05be0c8
heap_update: palloc the HOT-indexed tombstone scratch buffer
gburd May 8, 2026
fa86258
pruneheap: allow HOT-indexed tombstones in LP_UNUSED assertion
gburd May 8, 2026
03bb9ae
Remove hot_indexed_updates GUC; HOT-indexed is always on (catalogs ex…
gburd May 8, 2026
7711780
heap_update: reserve a 2nd LP slot when asking for a HOT-indexed-elig…
gburd May 8, 2026
684f1a7
Add hot_indexed_update_threshold GUC (percent cap on HOT-indexed elig…
gburd May 9, 2026
3609622
Add src/test/benchmarks/tepid: the HOT-indexed A/B harness
gburd May 9, 2026
b0e3acb
Expose HOT-indexed metrics: n_tup_hot_idx_upd counter + pg_relation_h…
gburd May 9, 2026
baa113f
Split hot_updates.sql into hot_updates + hot_indexed_updates
gburd May 10, 2026
e6bc1ae
Add README.HOT-INDEXED describing the feature
gburd May 10, 2026
819991e
Add index-key recheck in nodeIndexscan for HOT-indexed chains
gburd May 10, 2026
2be8b5e
Cap HOT-indexed chain length by relation geometry
gburd May 11, 2026
b4557d3
Compare real index keys on HOT-indexed unique check candidates
gburd May 11, 2026
a162a62
Standardise terminology: drop the "SIU" acronym throughout
gburd May 11, 2026
cd53bac
Teach logical decoding to strip the HOT-indexed tombstone trailer
gburd May 11, 2026
736f94c
Force non-HOT for UPDATEs applied by the logical replication worker
gburd May 11, 2026
b000168
README.HOT-INDEXED: document chain-length cap and logical-apply exemp…
gburd May 11, 2026
4b82be7
Narrow apply-path non-HOT exemption to subscribers with extra indexes
gburd May 11, 2026
b0b571f
check_exclusion_or_unique_constraint: recheck on HOT-indexed chain hits
gburd May 11, 2026
6b8b1f8
README.HOT-INDEXED: reflect narrowed apply-path and exclusion recheck
gburd May 11, 2026
cb132b8
Enable HOT-indexed updates on system catalogs
gburd May 11, 2026
a31ce52
README.HOT-INDEXED: document catalog enablement
gburd May 11, 2026
6a2bd14
Fix dangling-pointer bug in HOT-indexed WAL record emission
gburd May 12, 2026
4056521
Re-verify two-LP fit when RelationGetBufferForTuple returns same buffer
gburd May 12, 2026
e7c28d3
README.HOT-INDEXED: refine catalog enablement status
gburd May 12, 2026
024f813
bench/tepid: reset state between workloads
gburd May 12, 2026
50315f5
bench/tepid: record per-index sizes in the CSV output
gburd May 12, 2026
336aa34
bench/tepid: capture per-workload WAL record histograms
gburd May 12, 2026
85bf291
bench/tepid: separate classic-HOT and HOT-indexed counters
gburd May 12, 2026
b7ff4c3
bench/tepid: record clean A/B reference point
gburd May 12, 2026
4bd9e7f
bufpage: add PD_HAS_HOT_IDX_BRIDGES page-level flag
gburd May 12, 2026
048a7e9
hot_indexed: add bridge tombstone predicate and builder
gburd May 12, 2026
c3d6b9b
heap_hot_search_buffer: recognise HOT-indexed bridge tombstones
gburd May 12, 2026
9ab6500
heapam_xlog: reserve XLHP_HAS_HOT_IDX_BRIDGES flag
gburd May 12, 2026
a936734
pruneheap: write HOT-indexed bridge tombstones
gburd May 12, 2026
e803d8a
vacuumlazy: reclaim HOT-indexed bridge tombstones during index vacuum
gburd May 12, 2026
357ccca
pruneheap/vacuum: tombstone-bearing pages must not be all-visible
gburd May 12, 2026
7bc42b7
bench/tepid: record post-bridge A/B reference point
gburd May 12, 2026
409b7fc
amapi: add amrecheck_leaf_key callback; stop the nbtree leak in IOS
gburd May 12, 2026
290df8e
README.HOT-INDEXED: document bridge tombstones and all-visible fix
gburd May 12, 2026
6a3b760
Audit indexOK=false SeqScan callers for HOT-indexed safety
May 12, 2026
db7d46e
pgstat: add per-index HOT-indexed skipped/matched counters
May 12, 2026
e806ac3
heapam_xlog: reserve XLHP_HAS_PROMOTIONS flag
gburd May 12, 2026
b3eb48b
pruneheap: plumb HOT-indexed chain promotion through prune WAL
gburd May 12, 2026
a71f012
README.HOT-INDEXED: explain why chain promotion stays disabled
May 13, 2026
8dfd25c
pg_subscription: add subhotindexedmode column
May 12, 2026
e9154af
subscriptioncmds: parse hot_indexed_on_apply option
May 12, 2026
7745c6d
logical/worker: cache hot_indexed_on_apply mode
May 12, 2026
245f342
heapam: consult hot_indexed_on_apply mode on the apply path
May 12, 2026
321f3eb
doc: describe hot_indexed_on_apply subscription option
May 12, 2026
a1efa9b
subscription/039: TAP test for hot_indexed_on_apply
May 12, 2026
804b663
tablecmds: SeqScan pg_constraint conparentid in FK NOT ENFORCED recur…
May 13, 2026
09ffc36
heapam: refresh stale comments and harden tombstone trailer parser
May 13, 2026
15a25b9
heapam: replace 'fit check was too lax' panic message
May 13, 2026
5eae03b
hot_indexed: make HotIndexedBridgeSize a compile-time constant
May 13, 2026
29256ee
relcache: justify chain-cap heuristic constants in RelationGetHotInde…
May 13, 2026
84a901f
regress: extend hot_indexed_updates with chain-cap and reclamation cases
May 13, 2026
90f105f
relfilenumbermap: SeqScan pg_class in RelidByRelfilenumber
May 13, 2026
eac2ba1
Drop residual SIU acronym from struct fields, locals, and comments
May 13, 2026
6310a91
pruneheap: remove unused chain-promotion WAL flag and plumbing
gburd May 13, 2026
a0d5537
README.HOT-INDEXED: rewrite chain-promotion future-work section
gburd May 13, 2026
b0edd5c
executor: route HOT-indexed leaf-key recheck through amrecheck_leaf_key
May 13, 2026
342b53c
relcache: cache RelationHasExclusionConstraint result on Relation
May 13, 2026
2bb7f51
heapam: deduplicate index-attr bitmap fetches in HeapUpdateHotAllowable
May 13, 2026
29d22e7
Rename PD_HAS_HOT_IDX_BRIDGES to PD_HAS_HOT_INDEXED_BRIDGES
May 13, 2026
8790a68
Refresh README.HOT-INDEXED with crash, statistics, and tombstone details
May 13, 2026
426c6f3
tests: add recovery TAP for HOT-indexed bridge tombstones
May 13, 2026
5b319b9
pruneheap: bridge aborted HOT-indexed heap-only tuples
gburd May 13, 2026
aa178cc
pruneheap: bridge multi-update aborted HOT-indexed mid-chain tuples
gburd May 13, 2026
d0905c1
pruneheap: mark unbridgeable aborted HOT-indexed orphans LP_DEAD
gburd May 13, 2026
bd9aea9
Add isolation spec for prune-vs-reader on bridge transition
gburd May 13, 2026
268f6b7
nbtree: make _bt_heap_keys_equal_leaf private to nbtree
gburd May 13, 2026
79eb360
subscription/039: per-mode subscriber INSERT with stale leaf key
gburd May 13, 2026
c824349
Doc/comment refresh and SIU residue removal
gburd May 14, 2026
4483529
regress/cluster: pick up upstream REPACK HINT wording change
gburd May 14, 2026
485c533
bench/tepid: post-rebase wide_64 reference, 2026-05-14
gburd May 14, 2026
00629a5
relcache: add RelationGetIndexAttrBitmapNoCopy borrowing variant
gburd May 14, 2026
b9b1f53
bench/tepid: split HOT counters into classic/HOT-indexed/non-HOT
gburd May 14, 2026
24f7177
executor: skip slot-attr comparison when UPDATE targets no indexed col
gburd May 14, 2026
24ba068
vacuumlazy: track bridge count to skip post-vacuum rescan
gburd May 14, 2026
c70bbd3
amcheck: validate HOT-indexed tombstone items
May 14, 2026
9d8f92d
heap: skip KEY bitmap fetch in HeapUpdateDetermineLockmode for empty …
gburd May 14, 2026
63df3b8
pruneheap: reclaim adjacent tombstones whose target became a bridge
gburd May 14, 2026
b55f35a
regress/cluster: REPACK CONCURRENTLY HINT now emits parens
gburd May 14, 2026
2465226
executor: don't take wide_0 fast path for FOR PORTION OF or exclusion
gburd May 14, 2026
aad3b07
Rename pgstat counters and subscription column for upstream-style names
gburd May 14, 2026
1203572
bench/tepid: post-optimization wide_64 multi-pass reference, 2026-05-14
gburd May 14, 2026
25bb17f
pruneheap: union HOT-indexed tombstone bitmaps at chain collapse
gburd May 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .clangd
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Diagnostics:
MissingIncludes: None
InlayHints:
Enabled: true
ParameterNames: true
DeducedTypes: true
CompileFlags:
CompilationDatabase: build/ # Search build/ directory for compile_commands.json
Remove: [ -Werror ]
Add:
- -DDEBUG
- -DLOCAL
- -DPGDLLIMPORT=
- -DPIC
- -O2
- -Wall
- -Wcast-function-type
- -Wconversion
- -Wdeclaration-after-statement
- -Wendif-labels
- -Werror=vla
- -Wextra
- -Wfloat-equal
- -Wformat-security
- -Wimplicit-fallthrough=3
- -Wmissing-format-attribute
- -Wmissing-prototypes
- -Wno-format-truncation
- -Wno-sign-conversion
- -Wno-stringop-truncation
- -Wno-unused-const-variable
- -Wpointer-arith
- -Wshadow
- -Wshadow=compatible-local
- -fPIC
- -fexcess-precision=standard
- -fno-strict-aliasing
- -fvisibility=hidden
- -fwrapv
- -g
- -std=c11
- -I.
- -I../../../../src/include
2 changes: 2 additions & 0 deletions .envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
watch_file flake.nix
use flake
70 changes: 70 additions & 0 deletions .gdbinit
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# tepid dev additions: .gdbinit for debugging the HOT-indexed updates branch
#
# Usage:
# gdb -x .gdbinit <postgres-binary>
# source .gdbinit (from inside gdb)
#
# This file is tracked in the repo for developer convenience on the tepid
# branch. It is not intended for upstream consumption and is ignored when
# generating patches for the mailing list.
#
# What this file does:
# - Sources src/tools/gdb/tepid-helpers.py, which registers three
# commands: tepid-break, tepid-page, tepid-index.
# - Calls tepid-break immediately to install pending breakpoints in
# every function the branch adds or materially changes. Breakpoints
# are pending so the command is safe to run before symbols load (e.g.
# before attach).
#
# Breakpoints fall into four functional groups:
# Write path: heap_build_hot_indexed_tombstone,
# heap_hot_indexed_tombstone_attr_modified,
# HeapUpdateHotAllowable, heap_update
# WAL: heap_xlog_update
# Read path: heap_hot_search_buffer, _bt_heap_keys_equal_leaf,
# ExecSetIndexUnchanged, RelationGetIndexedAttrs,
# _bt_check_unique
# Prune: prune_handle_tombstones
# Stats: pg_relation_hot_indexed_stats
#
# To disable a specific breakpoint group temporarily use gdb's own
# "disable" / "enable" commands with the breakpoint numbers shown by
# "info breakpoints" after tepid-break runs.

# Keep a local repo-rooted path in sync with the worktree.
source src/tools/gdb/tepid-helpers.py

# Install the breakpoints. Pending mode keeps them queued until the
# postgres binary has loaded symbols.
set breakpoint pending on
tepid-break

# Convenience: print (col=val, ...) tuples, one per line.
set print pretty on
set print array on
set print union on

# Useful aliases that don't have command-class entries in tepid-helpers.
define tbreak
tepid-break
end
document tbreak
Alias for tepid-break. Installs pending breakpoints for every
function the tepid branch adds or materially changes.
end

define tpage
tepid-page $arg0 $arg1
end
document tpage
tpage RELNAME BLKNUM -- show HOT chains on a heap page. Wraps
tepid-page; identical argument syntax.
end

define tindex
tepid-index $arg0 $arg1
end
document tindex
tindex IDXNAME BLKNUM -- show btree leaf entries. Wraps
tepid-index; identical argument syntax.
end
18 changes: 18 additions & 0 deletions .github/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Node modules
scripts/ai-review/node_modules/
# Note: package-lock.json should be committed for reproducible CI/CD builds

# Logs
scripts/ai-review/cost-log-*.json
scripts/ai-review/*.log

# OS files
.DS_Store
Thumbs.db

# Editor files
*.swp
*.swo
*~
.vscode/
.idea/
163 changes: 163 additions & 0 deletions .github/DEV_SETUP_FIX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Dev Setup Commit Fix - Summary

**Date:** 2026-03-10
**Issue:** Sync workflow was failing because "dev setup" commits were detected as pristine master violations

## Problem

The sync workflow was rejecting the "dev setup v19" commit (e5aa2da496c) because it modifies files outside `.github/`. The original logic only allowed `.github/`-only commits, but didn't account for personal development environment commits.

## Solution

Updated sync workflows to recognize commits with messages starting with "dev setup" (case-insensitive) as allowed on master, in addition to `.github/`-only commits.

## Changes Made

### 1. Updated Sync Workflows

**Files modified:**
- `.github/workflows/sync-upstream.yml` (automatic hourly sync)
- `.github/workflows/sync-upstream-manual.yml` (manual sync)

**New logic:**
```bash
# Check for "dev setup" commits
DEV_SETUP_COMMITS=$(git log --format=%s upstream/master..origin/master | grep -i "^dev setup" | wc -l)

# Allow merge if:
# - Only .github/ changes, OR
# - Has "dev setup" commits
if [ "$COMMITS_AHEAD" -gt 0 ] && [ "$NON_GITHUB_CHANGES" -gt 0 ]; then
if [ "$DEV_SETUP_COMMITS" -eq 0 ]; then
# FAIL: Code changes outside .github/ that aren't dev setup
exit 1
else
# OK: Dev setup commits are allowed
continue merge
fi
fi
```

### 2. Created Policy Documentation

**New file:** `.github/docs/pristine-master-policy.md`

Documents the "mostly pristine" master policy:
- ✅ `.github/` commits allowed (CI/CD configuration)
- ✅ "dev setup ..." commits allowed (personal development environment)
- ❌ Code changes not allowed (must use feature branches)

## Current Commit Order

```
master:
1. 9a2b895daa0 - Complete Phase 3: Windows builds + fix sync (newest)
2. 1e6379300f8 - Add CI/CD automation: hourly sync, Bedrock AI review
3. e5aa2da496c - dev setup v19
4. 03facc1211b - upstream commits... (oldest)
```

**All three local commits will now be preserved during sync:**
- Commit 1: Modifies `.github/` ✅
- Commit 2: Modifies `.github/` ✅
- Commit 3: Named "dev setup v19" ✅

## Testing

After committing these changes, the next hourly sync should:
1. Detect 3 commits ahead of upstream (including the fix commit)
2. Recognize that they're all allowed (`.github/` or "dev setup")
3. Successfully merge upstream changes
4. Create merge commit preserving all local commits

**Verify manually:**
```bash
# Trigger manual sync
# Actions → "Sync from Upstream (Manual)" → Run workflow

# Check logs for:
# "✓ Found 1 'dev setup' commit(s) - will merge"
# "✓ Successfully merged upstream with local configuration"
```

## Future Updates

When updating your development environment:

```bash
# Make changes
git add .clangd flake.nix .vscode/ .idea/

# IMPORTANT: Start commit message with "dev setup"
git commit -m "dev setup v20: Update IDE and LSP configuration"

git push origin master
```

The sync will recognize this and preserve it during merges.

**Naming patterns recognized:**
- `dev setup v20` ✅
- `Dev setup: Update tools` ✅
- `DEV SETUP - New config` ✅
- `development environment changes` ❌ (doesn't start with "dev setup")

## Benefits

1. **No manual sync resolution needed** for dev environment updates
2. **Simpler workflow** - dev setup stays on master where it's convenient
3. **Clear policy** - documented what's allowed vs what requires feature branches
4. **Automatic detection** - sync workflow handles it all automatically

## What to Commit

```bash
git add .github/workflows/sync-upstream.yml
git add .github/workflows/sync-upstream-manual.yml
git add .github/docs/pristine-master-policy.md
git add .github/DEV_SETUP_FIX.md

git commit -m "Fix sync to allow 'dev setup' commits on master

The sync workflow was failing because the 'dev setup v19' commit
modifies files outside .github/. Updated workflows to recognize
commits with messages starting with 'dev setup' as allowed on master.

Changes:
- Detect 'dev setup' commits by message pattern
- Allow merge if commits are .github/ OR dev setup
- Update merge messages to reflect preserved changes
- Document pristine master policy

This allows personal development environment commits (IDE configs,
debugging tools, shell aliases, etc.) on master without violating
the pristine mirror policy.

See .github/docs/pristine-master-policy.md for details"

git push origin master
```

## Next Sync Expected Behavior

```
Before:
Upstream: A---B---C---D (latest upstream)
Master: A---B---C---X---Y---Z (X=CI/CD, Y=CI/CD, Z=dev setup)

Status: 3 commits ahead, 1 commit behind

After:
Master: A---B---C---X---Y---Z---M
\ /
D-------/

Where M = Merge commit preserving all local changes
```

All three local commits (CI/CD + dev setup) preserved! ✅

---

**Status:** Ready to commit and test
**Documentation:** See `.github/docs/pristine-master-policy.md`
Loading
Loading