Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion okf-bundle/documentation-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Confirm:

| Check | Requirement |
|-------|-------------|
| **Canonical location** | Each topic has one owning doc; others link to it ([agent command policy](testing/agent-command-policy.md) for **all** agent shell commands; [change authoring](testing/change-authoring-workflow.md) for workflow/gates/frozen tree; [change authoring § validation evidence (blocking)](testing/change-authoring-workflow.md#validation-evidence-blocking); [running e2e](testing/running-e2e.md) for e2e `yarn tests:*` detail — [agent rule](testing/running-e2e.md#agent-rule-read-first); [platform coverage gate](testing/running-e2e.md#platform-coverage-gate-blocking); [iteration vocabulary](testing/iteration-vocabulary.md) for term ids only; [change authoring § quality standards](testing/change-authoring-workflow.md#quality-standards) for the review-findings resolution rule and the intractable-limitation bar; [compare-types README § justification bar](../.github/scripts/compare-types/README.md#justification-bar) for firebase-js-sdk type/API drift justification; [coverage design § expectations](testing/coverage-design.md#coverage-expectations-policy); [coverage design § evidence package](testing/coverage-design.md#coverage-evidence-package) for coverage completion evidence; [documentation site maintenance](documentation-site-maintenance.md) for `docs.json`, TypeDoc, and legacy redirect audits; this file for doc/commit policy, etc.) |
| **Canonical location** | Each topic has one owning doc; others link to it ([agent command policy](testing/agent-command-policy.md) for **all** agent shell commands; [change authoring](testing/change-authoring-workflow.md) for workflow/gates/frozen tree; [change authoring § validation evidence (blocking)](testing/change-authoring-workflow.md#validation-evidence-blocking); [running e2e](testing/running-e2e.md) for e2e `yarn tests:*` detail — [agent rule](testing/running-e2e.md#agent-rule-read-first); [platform coverage gate](testing/running-e2e.md#platform-coverage-gate-blocking); [iteration vocabulary](testing/iteration-vocabulary.md) for term ids and gate field names (`coverage_evidence_gate`); [change authoring § quality standards](testing/change-authoring-workflow.md#quality-standards) for the review-findings resolution rule and the intractable-limitation bar; [compare-types README § justification bar](../.github/scripts/compare-types/README.md#justification-bar) for firebase-js-sdk type/API drift justification; [coverage design § expectations](testing/coverage-design.md#coverage-expectations-policy); [coverage design § evidence package](testing/coverage-design.md#coverage-evidence-package) for coverage completion evidence and anti-patterns; [documentation site maintenance](documentation-site-maintenance.md) for `docs.json`, TypeDoc, and legacy redirect audits; this file for doc/commit policy, etc.) |
| **DRY** | No duplicated procedures, policy paragraphs, or ephemeral snapshots outside work queues |
| **Link hygiene** | Cross-links resolve; indexes list canonical entry points |
| **Durability** | No ephemeral fields leaked into general reference docs |
Expand Down
15 changes: 9 additions & 6 deletions okf-bundle/testing/change-authoring-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,9 @@ E2e scope, pre-flight, and harness gate: [running e2e § agent rule](running-e2e
| Gate | Closes when |
|------|-------------|
| `implementation` | `implementation` work type complete — code plus **unit-focused**-tier checks green on **every required platform** when native bridge or embed path changed ([platform coverage gate](running-e2e.md#platform-coverage-gate-blocking)); [static analysis](validation-checklist.md#lint-and-formatting) green on the diff |
| `review` | `independent-review` complete — **area-focused**-tier checks green on frozen tree; applicable [validation checklist](validation-checklist.md) rows green (including static analysis); **every review finding resolved** ([§ quality standards](#quality-standards)) |
| `commit` | Durable commit exists for the item **after** prior gates closed with [recorded evidence](#validation-evidence-blocking) |
| `coverage_evidence` | [Coverage evidence package](coverage-design.md#coverage-evidence-package) attached with verdict line — **required** when frozen diff touches `packages/*/lib/**` or native bridge (`packages/*/{android,ios}/**`); otherwise `n/a` |
| `review` | `independent-review` complete — **area-focused**-tier checks green on frozen tree; applicable [validation checklist](validation-checklist.md) rows green (including static analysis); **`coverage_evidence` gate closed** when required; **every review finding resolved** ([§ quality standards](#quality-standards)) |
| `commit` | Durable commit exists for the item **after** `implementation`, `coverage_evidence` (when required), and `review` gates closed with [recorded evidence](#validation-evidence-blocking) |

**Trust rule:** Code on disk or in git with `review` still **open** is unverified until `independent-review` closes the gate.

Expand All @@ -114,7 +115,8 @@ Gates close **only** when **recorded evidence** shows the required validation ti
| Gate | Minimum evidence (record in work-queue notes or review handoff) |
|------|------------------------------------------------------------------|
| **`implementation`** | Prepare/tsc/jest **exit codes**; when native or macOS runtime touched: **e2e pass count per required platform** + log path (e.g. `/tmp/rnfb-e2e-*.log`); **`yarn lint` exit code 0**; when `docs/**` changed: **`yarn lint:markdown`** + **`yarn lint:spellcheck` exit code 0** |
| **`review`** | Frozen-tree re-run of area-focused checklist; **`yarn lint` exit code 0**; when `docs/**` in frozen diff: **`yarn lint:markdown`** + **`yarn lint:spellcheck` exit code 0**; **coverage evidence package** when native or `packages/*/lib/**` bridge code touched ([coverage design § evidence package](coverage-design.md#coverage-evidence-package)); compare:types row for touched registered package |
| **`coverage_evidence`** | When lib/native bridge touched: full [coverage evidence package](coverage-design.md#coverage-evidence-package) at `.agents/reports/<item>/coverage-evidence.md` (or work-queue notes) including **verdict line**; post-process native artifacts after fresh e2e ([coverage design § reading per-file](coverage-design.md#reading-per-file-coverage-locally)) |
| **`review`** | Frozen-tree re-run of area-focused checklist; **`yarn lint` exit code 0**; when `docs/**` in frozen diff: **`yarn lint:markdown`** + **`yarn lint:spellcheck` exit code 0**; **`coverage_evidence` gate closed** when required; compare:types row for touched registered package |
| **`commit`** | Prior gates closed **with evidence**; no `.only` / harness overrides staged |
| **Publication** (`git push`, force-push, PR refresh) | **`review` gate closed on the exact commits being published**; evidence still valid (no product edits since last area-focused run) |

Expand All @@ -124,7 +126,8 @@ Gates close **only** when **recorded evidence** shows the required validation ti

### Forbidden shortcuts

- **`git commit`** while the current work type's validation tier is incomplete or evidence is missing.
- **`git commit`** while the current work type's validation tier is incomplete or evidence is missing — including **`coverage_evidence` gate open** when lib/native bridge touched.
- **Jet NYC `text-summary` after narrowed e2e** as coverage proof for native or `packages/*/lib/**` changes — it measures remapped TS only; use Jacoco/lcov on **changed files** ([coverage design § anti-patterns](coverage-design.md#anti-patterns-not-coverage-evidence)).
- **`git push` / force-push / PR update** claiming remediation or review-green **without** fresh area-focused evidence after the last product edit on the published commits.
- **History rewrite** (rebase, amend stack) **without** re-running validation for the rewritten scope — prior green results are **invalid**.
- **Self-accepted** parity or coverage gaps — only [acceptable exceptions](#acceptable-exceptions) with user confirmation or intractability evidence in durable OKF (e.g. parity registry row).
Expand Down Expand Up @@ -235,8 +238,8 @@ On a **frozen tree**:
1. Revert all `.only`.
2. Keep area narrowing; run **area-focused**-tier e2e for loaded package spec(s) on [**every required platform**](running-e2e.md#platform-coverage-gate-blocking) (serial; pre-flight each run).
3. Run applicable [validation checklist](validation-checklist.md) rows — **blocking:** [static analysis § lint and formatting](validation-checklist.md#lint-and-formatting) (`yarn lint:js` on the frozen tree; markdown/spellcheck when docs touched); `yarn reference:api` when public surface changed. For packages registered in `compare:types`, `yarn compare:types` is a **blocking review gate**: the touched package must have zero undocumented or stale differences before `review_gate` closes. If the global command fails on unrelated registered packages, record/fix that drift in the work queue; do not treat an unrelated failure as permission to skip the touched package's type-parity check.
4. **Coverage evidence package** — **blocking** when `packages/*/lib/**` or native bridge sources in the frozen diff: produce and attach per [coverage design § evidence package](coverage-design.md#coverage-evidence-package); investigate every non-100% reachable line before closing `review`.
5. Outcome closes **review gate** or returns to **`implementation`**.
4. **Coverage evidence package** — **blocking** when `packages/*/lib/**` or native bridge sources in the frozen diff: after fresh area-focused e2e, run native post-process ([coverage design § reading per-file](coverage-design.md#reading-per-file-coverage-locally)); produce package per [coverage design § evidence package](coverage-design.md#coverage-evidence-package); close **`coverage_evidence` gate** with verdict line; investigate every non-100% reachable line. **Missing or stale package = serious finding** — same bar as missing e2e counts.
5. Outcome closes **`review` gate** (only after **`coverage_evidence` gate** when required) or returns to **`implementation`**.

Keep **`implementation`** and **`independent-review`** in separate passes ([§ frozen tree](#frozen-tree)).

Expand Down
34 changes: 33 additions & 1 deletion okf-bundle/testing/coverage-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,39 @@ Produce after fresh e2e on every required platform, then post-process native art

**Verdict line:** `100% on reachable touched lines` **or** `NOT 100%` with numbered gaps and disposition (fixed / intractable with evidence / user-accepted deferral).

Reviewers treat missing or stale coverage evidence as a **blocking** finding — same bar as missing e2e counts ([change authoring § validation evidence](change-authoring-workflow.md#validation-evidence-blocking)).
Record in `.agents/reports/<item>/coverage-evidence.md` (preferred) or work-queue notes. Orchestrators **must not** close `coverage_evidence_gate` or `review_gate` without this file when lib/native bridge is touched.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"without this file" - could this be clarified? Review says change to "this evidence package", but it could also be "without changing this file"?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just the sort of ambiguity that can lead to drift - good catch - I'm changing to "without recording the evidence in one of those locations"


Reviewers treat missing, stale, or NYC-only summaries as a **serious** finding ([change authoring § independent-review](change-authoring-workflow.md#independent-review)).

### Subagent return fields (native / lib bridge touched)

Blocking YAML (or equivalent table) before `coverage_evidence_gate` closes:

```yaml
coverage_verdict: "100% on reachable touched lines" | "NOT 100%"
coverage_artifacts:
jacoco_xml: path # Android; use lcov path for iOS native
timestamp: ISO-8601
touched_regions:
- file: packages/.../Foo.java
lines: "L10-L45"
line_pct: 100
branch_map:
- branch: "onComplete when still registered"
test: "Functions.e2e.js — streaming cancel race"
gaps: [] # or numbered: disposition fixed | intractable+evidence | user-accepted deferral
```

<a id="anti-patterns-not-coverage-evidence"></a>

### Anti-patterns (not coverage evidence)

| Looks like coverage | Why it is not |
|---------------------|---------------|
| Jet NYC `text-summary` after narrowed `:test-cover` | Remapped TS aggregate for loaded modules — does not report Jacoco/lcov on native bridge lines |
| Whole-package or whole-harness statement % | Signal is **per changed file/region** in the frozen diff |
| Stale Jacoco XML / lcov without matching e2e run | Post-process deletes raw artifacts; re-run e2e first ([§ stale coverage](#stale-coverage-data)) |
| E2e pass counts alone | Proves behaviour, not line/branch coverage on touched native code |

## Platform parity (pipeline and bridge code)

Expand Down
4 changes: 3 additions & 1 deletion okf-bundle/testing/iteration-vocabulary.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,13 @@ Work queues use these **field names** (values: `open` | `closed`):
| Field | Tracks |
|-------|--------|
| `implementation_gate` | `implementation` work type complete |
| `coverage_evidence_gate` | [Coverage evidence package](coverage-design.md#coverage-evidence-package) complete with verdict line when lib/native bridge touched; otherwise `n/a` (treat as closed) |
| `review_gate` | `independent-review` work type complete |
| `commit_gate` | Durable commit exists for the item **after** prior gates closed with [validation evidence](change-authoring-workflow.md#validation-evidence-blocking) |

What closes each gate, trust rules, and loop transitions: [change authoring § gates](change-authoring-workflow.md#gates).

`commit_gate` closes when a durable commit exists whose subject matches the row's `commit_subject`, **after** `implementation_gate` and `review_gate` closed with [validation evidence](change-authoring-workflow.md#validation-evidence-blocking).
`commit_gate` closes when a durable commit exists whose subject matches the row's `commit_subject`, **after** `implementation_gate`, `coverage_evidence_gate` (when required), and `review_gate` closed with [validation evidence](change-authoring-workflow.md#validation-evidence-blocking).

Items may also be marked **`blocked`** when a dependency gate is open elsewhere.

Expand All @@ -70,6 +71,7 @@ Ephemeral work queues may record:
| `validation_tier` | `unit-focused` \| `area-focused` \| `full` |
| `platform` | Optional scope (e.g. `ios`) |
| `implementation_gate` | `open` \| `closed` |
| `coverage_evidence_gate` | `open` \| `closed` \| `n/a` |
| `review_gate` | `open` \| `closed` |
| `commit_gate` | `open` \| `closed` |
| `commit_subject` | Planned or landed **first line** of the item's focused commit (Conventional Commits subject). Set **before** `git commit`; must match the commit that closes `commit_gate`. Do not record SHAs. |
Expand Down
4 changes: 2 additions & 2 deletions okf-bundle/testing/validation-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ Before closing **`implementation_gate`**, **`review_gate`**, **`commit_gate`**,
| lint (CI) | yarn lint | 0 | — |
| lint:markdown (CI docs) | yarn lint:markdown | 0 | when docs/** in diff |
| lint:spellcheck (CI docs) | yarn lint:spellcheck | 0 | when docs/** in diff |
| coverage | post-process + region table | — | see coverage-design § evidence package |
| coverage | post-process + region table | — | [coverage evidence package](coverage-design.md#coverage-evidence-package); closes `coverage_evidence_gate` when lib/native touched |
```

**History rewrite invalidates** prior rows — re-run and replace the table after amend/rebase.
Expand All @@ -160,7 +160,7 @@ Before closing **`implementation_gate`**, **`review_gate`**, **`commit_gate`**,
- [ ] `yarn lint` (CI Lint job); `yarn lint:markdown` + `yarn lint:spellcheck` when `docs/**` changed
- [ ] E2e green on **every required platform** for the changed module ([platform coverage gate](running-e2e.md#platform-coverage-gate-blocking); [harness narrowing gate](running-e2e.md#harness-narrowing-gate-blocking); no `.only`; committed `RNFBDebug` remains `false`)
- [ ] [Validation evidence package](validation-checklist.md#validation-evidence-package) recorded (exit codes, e2e counts, log paths)
- [ ] [Coverage evidence package](coverage-design.md#coverage-evidence-package) when lib/native bridge touched — gaps investigated to fix, delete, or acceptable-exception bar
- [ ] [Coverage evidence package](coverage-design.md#coverage-evidence-package) when lib/native bridge touched — `coverage_evidence_gate` closed with verdict line; gaps investigated to fix, delete, or acceptable-exception bar
- [ ] OKF bundle reviewed/updated per § above

Package workflows may add items (e.g. pipeline before/after snapshots — [pipeline workflow](../packages/firestore/pipeline-implementation-workflow.md)).
Loading
Loading