Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,25 @@ node scripts/run-tests.mjs --filter "widget"
2. If changing runtime, layout, or renderer code, also run integration tests.
3. Run the full suite before committing.

### Mandatory Live PTY Validation for UI Regressions

For rendering/layout/theme regressions, do not stop at unit snapshots. Run the
app in a real PTY and collect frame audit evidence yourself before asking a
human to reproduce.

Canonical runbook:

- [`docs/dev/live-pty-debugging.md`](docs/dev/live-pty-debugging.md)

Minimum required checks for UI regression work:

1. Run target app/template in PTY with deterministic viewport.
2. Exercise relevant routes/keys (for starship: `1..6`, `t`, `q`).
3. Capture `REZI_FRAME_AUDIT` logs and analyze with
`node scripts/frame-audit-report.mjs ... --latest-pid`.
4. Capture app-level debug snapshots (`REZI_STARSHIP_DEBUG=1`) when applicable.
5. Include concrete evidence in your report (hash changes, route summary, key stages).

## Verification Protocol (Two-Agent Verification)

When verifying documentation or code changes, split into two passes:
Expand Down
9 changes: 9 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -505,6 +505,15 @@ result.toText(); // Render to plain text for snapshots

Test runner: `node:test`. Run all tests with `node scripts/run-tests.mjs`.

For rendering regressions, add a live PTY verification pass and frame-audit
evidence (not just snapshot/unit tests). Use:

- [`docs/dev/live-pty-debugging.md`](docs/dev/live-pty-debugging.md)

This runbook covers deterministic viewport setup, worker-mode PTY execution,
route/theme key driving, and cross-layer log analysis (`REZI_FRAME_AUDIT`,
`REZI_STARSHIP_DEBUG`, `frame-audit-report.mjs`).

## Skills (Repeatable Recipes)

Project-level skills for both Claude Code and Codex:
Expand Down
176 changes: 176 additions & 0 deletions docs/dev/live-pty-debugging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
# Live PTY UI Testing and Frame Audit Runbook

This runbook documents how to validate Rezi UI behavior autonomously in a real
terminal (PTY), capture end-to-end frame telemetry, and pinpoint regressions
across core/node/native layers.

Use this before asking a human for screenshots.

## Why this exists

Headless/unit tests catch many issues, but rendering regressions often involve:

- terminal dimensions and capability negotiation
- worker transport boundaries (core -> node worker -> native)
- partial redraw/damage behavior across many frames

The PTY + frame-audit workflow gives deterministic evidence for all of those.

## Prerequisites

From repo root:

```bash
cd <repo-root>
npx tsc -b packages/core packages/node packages/create-rezi
```

## Canonical interactive run (Starship template)

This enables:

- app-level debug snapshots (`REZI_STARSHIP_DEBUG`)
- cross-layer frame audit (`REZI_FRAME_AUDIT`)
- worker execution path (`REZI_STARSHIP_EXECUTION_MODE=worker`)

```bash
cd <repo-root>
: > /tmp/rezi-frame-audit.ndjson
: > /tmp/starship.log

env -u NO_COLOR \
REZI_STARSHIP_EXECUTION_MODE=worker \
REZI_STARSHIP_DEBUG=1 \
REZI_STARSHIP_DEBUG_LOG=/tmp/starship.log \
REZI_FRAME_AUDIT=1 \
REZI_FRAME_AUDIT_LOG=/tmp/rezi-frame-audit.ndjson \
npx tsx packages/create-rezi/templates/starship/src/main.ts
```

Key controls in template:

- `1..6`: route switch (bridge/engineering/crew/comms/cargo/settings)
- `t`: cycle theme
- `q`: quit

## Deterministic viewport (important)

Many regressions are viewport-threshold dependent. Always test with a known
size before comparing runs.

For an interactive shell/PTY:

```bash
stty rows 68 cols 300
```

Then launch the app in that same PTY.

## Autonomous PTY execution (agent workflow)

When your agent runtime supports PTY stdin/stdout control:

1. Start app in PTY mode (with env above).
2. Send key sequences (`2`, `3`, `t`, `q`) through stdin.
3. Wait between keys to allow frames to settle.
4. Quit and analyze logs.

Do not rely only on static test snapshots for visual regressions.

## Frame audit analysis

Use the built-in analyzer:

```bash
node scripts/frame-audit-report.mjs /tmp/rezi-frame-audit.ndjson --latest-pid
```

What to look for:

- `backend_submitted`, `worker_payload`, `worker_accepted`, `worker_completed`
should stay aligned in worker mode.
- `hash_mismatch_backend_vs_worker` should be `0`.
- `top_opcodes` should reflect expected widget workload.
- `route_summary` should show submissions for every exercised route.
- `native_summary_records`/`native_header_records` confirm native debug pull
from worker path.

If a log contains multiple app runs, always use `--latest-pid` (or `--pid=<n>`)
to avoid mixed-session confusion.

## Useful grep patterns

```bash
rg "runtime.command|runtime.fatal|shell.layout|engineering.layout|engineering.render|crew.render" /tmp/starship.log
rg "\"stage\":\"table.layout\"|\"stage\":\"drawlist.built\"|\"stage\":\"frame.submitted\"|\"stage\":\"frame.completed\"" /tmp/rezi-frame-audit.ndjson
```

## Optional deep capture (drawlist bytes)

Capture raw drawlist payload snapshots for diffing:

```bash
env \
REZI_FRAME_AUDIT=1 \
REZI_FRAME_AUDIT_DUMP_DIR=/tmp/rezi-drawlist-dumps \
REZI_FRAME_AUDIT_DUMP_MAX=20 \
REZI_FRAME_AUDIT_DUMP_ROUTE=crew \
npx tsx packages/create-rezi/templates/starship/src/main.ts
```

This writes paired `.bin` + `.json` files with hashes and metadata.

## Native trace through frame-audit

Native debug records are enabled by frame audit in worker mode. Controls:

- `REZI_FRAME_AUDIT_NATIVE=1|0` (default on when frame audit is enabled)
- `REZI_FRAME_AUDIT_NATIVE_RING=<bytes>` (ring size override)

Look for stages such as:

- `native.debug.header`
- `native.drawlist.summary`
- `native.frame.*`
- `native.perf.*`

## Triage playbook for common regressions

### 1) “Theme only updates animated region”

Check:

1. `runtime.command` contains `cycle-theme`.
2. `drawlist.built` hashes change after theme switch.
3. `frame.submitted`/`frame.completed` continue for that route.

If hashes do not change, bug is likely in view/theme resolution.
If hashes change but screen does not, investigate native diff/damage path.

### 2) “Table looks empty or only one row visible”

Check `table.layout` record:

- `bodyH`
- `visibleRows`
- `startIndex` / `endIndex`
- table rect height

If `bodyH` is too small, inspect parent layout/flex and sibling widgets
(pagination or controls often steal height).

### 3) “Worker mode renders differently from inline”

Run both modes with identical viewport and compare audit summaries:

- worker: `REZI_STARSHIP_EXECUTION_MODE=worker`
- inline: `REZI_STARSHIP_EXECUTION_MODE=inline`

If only worker diverges, focus on backend transport and worker audit stages.

## Guardrails

- Keep all instrumentation opt-in via env vars.
- Never print continuous debug spam to stdout during normal app usage.
- Write logs to files (`/tmp/...`) and inspect post-run.
- Prefer deterministic viewport + scripted route/theme steps when verifying fixes.
12 changes: 12 additions & 0 deletions docs/dev/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,18 @@ npm run test:e2e
npm run test:e2e:reduced
```

## Live PTY Rendering Validation (for UI regressions)

For terminal rendering/theme/layout regressions, run a live PTY session with
frame-audit instrumentation in addition to normal tests.

Use the dedicated runbook:

- [Live PTY UI Testing and Frame Audit Runbook](live-pty-debugging.md)

That guide includes deterministic viewport setup, worker-mode run commands,
scripted key driving, and cross-layer telemetry analysis.

## Test Categories

### Unit Tests
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ nav:
- Repo Layout: dev/repo-layout.md
- Build: dev/build.md
- Testing: dev/testing.md
- Live PTY Debugging: dev/live-pty-debugging.md
- Code Standards: dev/code-standards.md
- Ink Compat Debugging: dev/ink-compat-debugging.md
- Perf Regressions: dev/perf-regressions.md
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
truncateWithEllipsis,
} from "../../../layout/textMeasure.js";
import type { Rect } from "../../../layout/types.js";
import { FRAME_AUDIT_ENABLED, emitFrameAudit } from "../../../perf/frameAudit.js";
import type { RuntimeInstance } from "../../../runtime/commit.js";
import type { FocusState } from "../../../runtime/focus.js";
import type {
Expand Down Expand Up @@ -605,6 +606,32 @@ export function renderCollectionWidget(
? Math.min(rowCount, startIndex + visibleRows + overscan)
: rowCount;

if (FRAME_AUDIT_ENABLED) {
emitFrameAudit(
"tableWidget",
"table.layout",
Object.freeze({
tableId: props.id,
x: rect.x,
y: rect.y,
w: rect.w,
h: rect.h,
innerW,
innerH,
bodyY,
bodyH,
rowCount,
headerHeight,
rowHeight: safeRowHeight,
virtualized,
startIndex,
endIndex,
visibleRows,
overscan,
}),
);
}

if (tableStore) {
tableStore.set(props.id, { viewportHeight: bodyH, startIndex, endIndex });
}
Expand Down
5 changes: 4 additions & 1 deletion packages/core/src/widgets/__tests__/pagination.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,10 @@ describe("pagination ids and vnode", () => {
const zoneNode = children[0];
assert.equal(zoneNode?.kind, "focusZone");
if (zoneNode?.kind !== "focusZone") return;
const ids = zoneNode.children
const controlsRow = zoneNode.children[0];
assert.equal(controlsRow?.kind, "row");
if (controlsRow?.kind !== "row") return;
const ids = controlsRow.children
.filter((child) => child.kind === "button")
.map((child) => (child.kind === "button" ? child.props.id : ""));
assert.equal(ids.includes(getPaginationControlId("pages", "first")), true);
Expand Down
12 changes: 11 additions & 1 deletion packages/core/src/widgets/pagination.ts
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,16 @@ export function buildPaginationChildren(props: PaginationProps): readonly VNode[
});
}

const controlsRow: VNode = {
kind: "row",
props: {
gap: 0,
wrap: true,
items: "center",
},
children: Object.freeze(controls),
};

const zone: VNode = {
kind: "focusZone",
props: {
Expand All @@ -252,7 +262,7 @@ export function buildPaginationChildren(props: PaginationProps): readonly VNode[
columns: 1,
wrapAround: false,
},
children: Object.freeze(controls),
children: Object.freeze([controlsRow]),
};

return Object.freeze([zone]);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -554,7 +554,7 @@ export function renderBridgeScreen(
title: "Bridge Overview",
context,
deps,
body: ui.column({ gap: SPACE.sm, width: "100%" }, [
body: ui.column({ gap: SPACE.sm, width: "100%", height: "100%" }, [
BridgeCommandDeck({ key: "bridge-command-deck", state, dispatch: deps.dispatch }),
]),
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ export function renderCargoScreen(
title: "Cargo Hold",
context,
deps,
body: ui.column({ gap: SPACE.sm, width: "100%" }, [
body: ui.column({ gap: SPACE.sm, width: "100%", height: "100%" }, [
CargoDeck({
key: "cargo-deck",
state: context.state,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,7 @@ export function renderCommsScreen(
title: "Communications",
context,
deps,
body: ui.column({ gap: SPACE.sm, width: "100%" }, [
body: ui.column({ gap: SPACE.sm, width: "100%", height: "100%" }, [
CommsDeck({
key: "comms-deck",
state: context.state,
Expand Down
Loading
Loading