Skip to content
Draft

Undo #21

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
43 changes: 43 additions & 0 deletions .clangd
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Diagnostics:
MissingIncludes: None
InlayHints:
Enabled: true
ParameterNames: true
DeducedTypes: true
CompileFlags:
CompilationDatabase: build/ # Search build/ directory for compile_commands.json
Remove: [ -Werror ]
Add:
- -DDEBUG
- -DLOCAL
- -DPGDLLIMPORT=
- -DPIC
- -O2
- -Wall
- -Wcast-function-type
- -Wconversion
- -Wdeclaration-after-statement
- -Wendif-labels
- -Werror=vla
- -Wextra
- -Wfloat-equal
- -Wformat-security
- -Wimplicit-fallthrough=3
- -Wmissing-format-attribute
- -Wmissing-prototypes
- -Wno-format-truncation
- -Wno-sign-conversion
- -Wno-stringop-truncation
- -Wno-unused-const-variable
- -Wpointer-arith
- -Wshadow
- -Wshadow=compatible-local
- -fPIC
- -fexcess-precision=standard
- -fno-strict-aliasing
- -fvisibility=hidden
- -fwrapv
- -g
- -std=c11
- -I.
- -I../../../../src/include
2 changes: 2 additions & 0 deletions .envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
watch_file flake.nix
use flake
156 changes: 156 additions & 0 deletions .gdbinit
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# HOT Indexed Updates — GDB breakpoints for code review
#
# Usage: gdb -x .gdbinit <postgres-binary>
# Or from gdb: source .gdbinit
#
# These breakpoints cover the major code paths introduced or modified by
# the HOT indexed updates patch series. They are organized by subsystem
# to make it easy to enable/disable groups during debugging.
#
# Tip: to skip to a specific subsystem, disable all then enable selectively:
# disable breakpoints
# enable 1 2 3 # just the update-decision group

# =========================================================================
# 1. UPDATE DECISION — heap_update() HOT/HOT-indexed/non-HOT choice
# src/backend/access/heap/heapam.c
# =========================================================================

# Main entry: heap_update
break heapam.c:3210

# HOT decision block: pure HOT vs HOT indexed vs non-HOT
# Line 4019: pure HOT (no indexed columns changed)
# Line 4024: HOT indexed path (non-catalog, some indexed columns changed)
# Line 4031: predict augmented tuple size
# Line 4033: size+space check before creating augmented tuple
break heapam.c:4019
break heapam.c:4024
break heapam.c:4033

# Set HEAP_INDEXED_UPDATED flag on new tuple before page insertion
break heapam.c:4101

# Restore HEAP_INDEXED_UPDATED on old tuple (only if it previously had it)
break heapam.c:4147

# =========================================================================
# 2. TUPLE CREATION — building the augmented tuple with embedded bitmap
# src/backend/access/heap/heapam.c
# =========================================================================

# Predict augmented tuple size (returns 0 if t_hoff would overflow)
break heap_hot_indexed_tuple_size

# Create augmented tuple with embedded modified-column bitmap
break heap_hot_indexed_create_tuple

# Serialize Bitmapset into raw bytes in tuple header
break heap_hot_indexed_serialize_bitmap

# =========================================================================
# 3. BITMAP UTILITIES — raw bitmap operations for chain following
# src/backend/access/heap/heapam.c
# =========================================================================

# Compute raw bitmap byte size from natts
break heap_hot_indexed_bitmap_raw_size

# Check if tuple header has room for bitmap between null bitmap and data
break heap_hot_indexed_has_bitmap_space

# Read HOT indexed bitmap from tuple header (returns Bitmapset)
break heap_hot_indexed_read_bitmap

# Fast overlap check: does tuple's raw bitmap overlap with indexed_attrs?
break heap_hot_indexed_bitmap_overlaps_raw

# OR a tuple's raw bitmap into an accumulator buffer
break heap_hot_indexed_bitmap_or_raw

# Check if accumulated raw bitmap overlaps with indexed_attrs
break heap_hot_indexed_accum_overlaps

# Merge bitmaps from dead tuples into a target tuple on the page
break heap_hot_indexed_merge_bitmaps_raw

# Deserialize raw bytes back to Bitmapset
break heap_hot_indexed_deserialize_bitmap

# =========================================================================
# 4. INDEX SCAN — HOT chain following with stale-entry detection
# src/backend/access/heap/heapam_indexscan.c
# =========================================================================

# Main HOT chain search with indexed update awareness
break heap_hot_search_buffer

# Redirect-with-data: initialize bitmap accumulator from collapsed redirect
break heapam_indexscan.c:182

# Accumulate bitmap from INDEXED_UPDATED tuple in chain
break heapam_indexscan.c:250

# Stale entry detection: accumulated bitmap overlaps this index's attrs
break heapam_indexscan.c:297

# =========================================================================
# 5. INDEX SCAN SETUP — indexed_attrs bitmap computation
# src/backend/access/index/indexam.c
# =========================================================================

# Compute indexed_attrs for HOT indexed update chain following
break indexam.c:299

# =========================================================================
# 6. INDEX INSERTION — skip unchanged indexes for HOT indexed updates
# src/backend/executor/execIndexing.c
# =========================================================================

# Entry: insert/update index tuples
break ExecInsertIndexTuples

# Index skip decision: skip indexes whose attrs don't overlap modified set
break execIndexing.c:370

# =========================================================================
# 7. PRUNING — chain collapsing and redirect-with-data
# src/backend/access/heap/pruneheap.c
# =========================================================================

# Main prune function
break heap_page_prune_and_freeze

# Per-chain pruning entry
break heap_prune_chain

# Chain collapsing: collect bitmaps from dead INDEXED_UPDATED intermediates
break pruneheap.c:1802

# OR dead tuple bitmaps into combined bitmap
break pruneheap.c:1836

# Record redirect-with-data for execute phase
break pruneheap.c:1863

# Execute phase: apply redirect-with-data entries on the page
break pruneheap.c:1287

# =========================================================================
# 8. WAL REPLAY — recovery of HOT indexed updates
# src/backend/access/heap/heapam_xlog.c
# =========================================================================

# WAL replay for XLOG_HEAP2_INDEXED_UPDATE
break heap_xlog_indexed_update

# =========================================================================
# 9. WAL LOGGING — writing HOT indexed update records
# src/backend/access/heap/heapam.c
# =========================================================================

# WAL logging for heap updates (handles indexed_update flag)
break log_heap_update

# Serialize redirect-with-data into WAL record (pruneheap.c)
break pruneheap.c:2936
18 changes: 18 additions & 0 deletions .github/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Node modules
scripts/ai-review/node_modules/
# Note: package-lock.json should be committed for reproducible CI/CD builds

# Logs
scripts/ai-review/cost-log-*.json
scripts/ai-review/*.log

# OS files
.DS_Store
Thumbs.db

# Editor files
*.swp
*.swo
*~
.vscode/
.idea/
163 changes: 163 additions & 0 deletions .github/DEV_SETUP_FIX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Dev Setup Commit Fix - Summary

**Date:** 2026-03-10
**Issue:** Sync workflow was failing because "dev setup" commits were detected as pristine master violations

## Problem

The sync workflow was rejecting the "dev setup v19" commit (e5aa2da496c) because it modifies files outside `.github/`. The original logic only allowed `.github/`-only commits, but didn't account for personal development environment commits.

## Solution

Updated sync workflows to recognize commits with messages starting with "dev setup" (case-insensitive) as allowed on master, in addition to `.github/`-only commits.

## Changes Made

### 1. Updated Sync Workflows

**Files modified:**
- `.github/workflows/sync-upstream.yml` (automatic hourly sync)
- `.github/workflows/sync-upstream-manual.yml` (manual sync)

**New logic:**
```bash
# Check for "dev setup" commits
DEV_SETUP_COMMITS=$(git log --format=%s upstream/master..origin/master | grep -i "^dev setup" | wc -l)

# Allow merge if:
# - Only .github/ changes, OR
# - Has "dev setup" commits
if [ "$COMMITS_AHEAD" -gt 0 ] && [ "$NON_GITHUB_CHANGES" -gt 0 ]; then
if [ "$DEV_SETUP_COMMITS" -eq 0 ]; then
# FAIL: Code changes outside .github/ that aren't dev setup
exit 1
else
# OK: Dev setup commits are allowed
continue merge
fi
fi
```

### 2. Created Policy Documentation

**New file:** `.github/docs/pristine-master-policy.md`

Documents the "mostly pristine" master policy:
- ✅ `.github/` commits allowed (CI/CD configuration)
- ✅ "dev setup ..." commits allowed (personal development environment)
- ❌ Code changes not allowed (must use feature branches)

## Current Commit Order

```
master:
1. 9a2b895daa0 - Complete Phase 3: Windows builds + fix sync (newest)
2. 1e6379300f8 - Add CI/CD automation: hourly sync, Bedrock AI review
3. e5aa2da496c - dev setup v19
4. 03facc1211b - upstream commits... (oldest)
```

**All three local commits will now be preserved during sync:**
- Commit 1: Modifies `.github/` ✅
- Commit 2: Modifies `.github/` ✅
- Commit 3: Named "dev setup v19" ✅

## Testing

After committing these changes, the next hourly sync should:
1. Detect 3 commits ahead of upstream (including the fix commit)
2. Recognize that they're all allowed (`.github/` or "dev setup")
3. Successfully merge upstream changes
4. Create merge commit preserving all local commits

**Verify manually:**
```bash
# Trigger manual sync
# Actions → "Sync from Upstream (Manual)" → Run workflow

# Check logs for:
# "✓ Found 1 'dev setup' commit(s) - will merge"
# "✓ Successfully merged upstream with local configuration"
```

## Future Updates

When updating your development environment:

```bash
# Make changes
git add .clangd flake.nix .vscode/ .idea/

# IMPORTANT: Start commit message with "dev setup"
git commit -m "dev setup v20: Update IDE and LSP configuration"

git push origin master
```

The sync will recognize this and preserve it during merges.

**Naming patterns recognized:**
- `dev setup v20` ✅
- `Dev setup: Update tools` ✅
- `DEV SETUP - New config` ✅
- `development environment changes` ❌ (doesn't start with "dev setup")

## Benefits

1. **No manual sync resolution needed** for dev environment updates
2. **Simpler workflow** - dev setup stays on master where it's convenient
3. **Clear policy** - documented what's allowed vs what requires feature branches
4. **Automatic detection** - sync workflow handles it all automatically

## What to Commit

```bash
git add .github/workflows/sync-upstream.yml
git add .github/workflows/sync-upstream-manual.yml
git add .github/docs/pristine-master-policy.md
git add .github/DEV_SETUP_FIX.md

git commit -m "Fix sync to allow 'dev setup' commits on master

The sync workflow was failing because the 'dev setup v19' commit
modifies files outside .github/. Updated workflows to recognize
commits with messages starting with 'dev setup' as allowed on master.

Changes:
- Detect 'dev setup' commits by message pattern
- Allow merge if commits are .github/ OR dev setup
- Update merge messages to reflect preserved changes
- Document pristine master policy

This allows personal development environment commits (IDE configs,
debugging tools, shell aliases, etc.) on master without violating
the pristine mirror policy.

See .github/docs/pristine-master-policy.md for details"

git push origin master
```

## Next Sync Expected Behavior

```
Before:
Upstream: A---B---C---D (latest upstream)
Master: A---B---C---X---Y---Z (X=CI/CD, Y=CI/CD, Z=dev setup)

Status: 3 commits ahead, 1 commit behind

After:
Master: A---B---C---X---Y---Z---M
\ /
D-------/

Where M = Merge commit preserving all local changes
```

All three local commits (CI/CD + dev setup) preserved! ✅

---

**Status:** Ready to commit and test
**Documentation:** See `.github/docs/pristine-master-policy.md`
Loading
Loading