diff --git a/.gitignore b/.gitignore index e2700c0..1001a2f 100644 --- a/.gitignore +++ b/.gitignore @@ -3,9 +3,25 @@ /modules /prompts /.claude +.serena IDEAS.md node_modules/ *.log *.jsonl .mcp.json CLAUDE.md +/.pos-supervisor + +# Test artifacts — integration tests write to fixture .pos-supervisor dirs +**/.pos-supervisor/sessions/ +**/.pos-supervisor/blobs/ +**/.pos-supervisor/analytics.db +**/.pos-supervisor/analytics.db-wal +**/.pos-supervisor/analytics.db-shm + +# Stray test/scratch files +/t +/2026-*.txt +.serena/project.yml +.gitignore~ +.serena/project.yml diff --git a/CHANGELOG.md b/CHANGELOG.md index 8e81122..13e4c36 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,1002 @@ # Changelog +## 0.7.3 — 2026-04-30 + +Reporting baseline + sample-size-gated labels — operator-set checkpoint +that filters every dashboard widget and exported Markdown report by a +chosen "stats since" timestamp, plus a presentation-layer label gate +that replaces nonsense `AT RISK -100%` / `HARMFUL` headlines on N<5 +samples with `INSUFFICIENT_DATA`. Engine state (auto-disable, case-base +scoring, CAC predictor, adaptive-mode probation) is **never** baselined +— it always sees full history. Default behaviour with no baseline set +is identical to 0.7.2. + +### Added — `src/core/analytics-store.js` baseline helpers + +Two new meta keys (`analytics_baseline_ts`, `analytics_baseline_set_at`) +plus four helpers: `getBaselineTs()`, `getBaselineMeta()`, +`setBaselineTs(iso | null)`, `clearBaseline()`. Stored in the existing +`meta` table — no schema migration. `setBaselineTs` validates ISO input +and rejects malformed strings with `TypeError` so the HTTP layer can +return 400 cleanly. The baseline survives `rebuild()` (rebuild only +clears derived data, not meta). + +### Added — `src/core/analytics-labels.js` + +Pure, side-effect-free presentation module owning the GOOD / OK / LOW / +HARMFUL, AT RISK / UNMATCHED, and INSUFFICIENT_DATA labels. The +`LABEL_MIN_OUTCOMES = 5` gate is the load-bearing change: a rule with a +single regression no longer headlines as AT RISK -100%, it lands in +INSUFFICIENT_DATA. Exports `checkLabel`, `ruleLabel`, `harmfulSummary`, +`withCheckLabels`, `withRuleLabels`. The HTTP layer wraps every +scorecard / rule-performance response with `withCheckLabels` / +`withRuleLabels` so the dashboard reads `.label` directly without +recomputing client-side. Inline label calculations in dashboard.js are +preserved as fallbacks for un-labelled responses. + +### Added — `since` parameter across reporting queries + +Tri-state contract on every reporting query in +`src/core/analytics-queries.js` (`checkScorecards`, `rulePerformance`, +`fixRulePerformance`, `fixAdoptionFunnel`, `knowledgeGaps`, +`confidenceCalibration`, `ruleScoresByCategory`, `sessionSummaries`, +`toolSequenceBigrams`, `diagnosticJourney`, `ruleDrilldown`, +`recommendations`) and the reporting paths in `src/core/case-base.js` +(`retrieveCases`, `retrieveCasesByCheck`, `ruleScores`, `suggestedRules`, +`synthesizeGuardPredicate`): + +- `since: undefined` → reads the operator-set baseline from + `meta.analytics_baseline_ts`. Absent meta ⇒ no filter ⇒ full history. + This is the dashboard / report default. +- `since: null` → explicit bypass. Reserved for engine-state callers + that must see full history regardless of baseline. + `server.js:syncDisabledRules` and `tools/server-status.js` were + updated to pass this. `scoreRule` and `cac-predictor` providers and + `resolveProbation` keep no `since` parameter at all — they cannot + accept a baseline argument by design. +- `since: ''` → explicit override. Used by the dashboard's "Stats + since" dropdown for 24h / 7d / custom selections. + +### Added — HTTP endpoints + `?since=` parameter + +Two new endpoints on `src/http-server.js`: + +- `GET /api/analytics/baseline` → `{ baseline_ts, set_at }`. +- `POST /api/analytics/baseline` body `{ baseline_ts: ISO | null }` → + sets / clears, echoes the resolved meta. 400 on malformed ISO. + +Every existing analytics endpoint accepts `?since=` (explicit +override), `?since=all` (engine bypass), or omits `since` (meta +default). The exported `parseSinceParam` helper has unit-test coverage +pinning the tri-state contract. Responses include a `since` echo field +so the dashboard renders the "Stats since" status pill without a +separate roundtrip. + +### Added — Dashboard "Stats since" controls + +The Analytics tab's refresh bar gains a select + "Set baseline now" / +"Clear baseline" buttons + an inline state pill. The dropdown's +"Since baseline (default)" option mirrors the report's behaviour; +"All time" bypasses; "Last 24 hours" / "Last 7 days" / "Custom" do what +they say. Custom takes a free-form ISO string. Setting / clearing the +baseline triggers a full analytics refresh so every widget reflects the +change. The Markdown report header gains a "Stats since: …" field that +echoes whichever filter the report was generated under, so an old +export remains self-documenting. + +### Changed — Sample-size gate replaces inline label calcs + +Three Markdown-report rendering sites in `src/dashboard.js` (executive +summary HARMFUL list, scorecard table, rule-performance table) and the +live HTML rule-performance table now read `.label` from the server +response and fall back to inline calculations only when the server +didn't attach one. The previous behaviour of computing labels from raw +effectiveness without a sample-size guard is gone. + +### Engine state — explicit bypass + +`src/server.js:syncDisabledRules` and `src/tools/server-status.js` +auto-disable / disabled-rules snapshot now pass `since: null` +explicitly. `resolveSince` recognises `null` as the engine bypass +marker and returns it unchanged regardless of any operator baseline. +This keeps the auto-disable loop and case-base scoring stable across +baseline edits — operators can experiment with reporting windows +without affecting the runtime engine state. + +### Tests + +- `tests/unit/analytics-store.test.js` — 7 new tests covering + baseline get / set / clear / persistence / rebuild-survival / + validation. +- `tests/unit/analytics-labels.test.js` (new) — 27 tests pinning the + label contract, the sample-size gate at the threshold boundary, the + `unmatched` precedence, and the `withCheckLabels` / `withRuleLabels` + immutability. +- `tests/unit/analytics-queries.test.js` — 14 new `since`-variant + tests, one per filterable function plus precedence cases. +- `tests/unit/case-base.test.js` — 8 new tests covering the case-base + reporting paths plus a deliberate test that `scoreRule` has no + `since` parameter (engine-path invariant). +- `tests/unit/http-since-param.test.js` (new) — 8 tests pinning the + HTTP-layer `?since=` parser tri-state contract. + +Total: 64 new tests. Full unit suite passes (1 pre-existing failure +in `load-development-guide` expectation drift, unrelated to this +change). + +## 0.7.2 — 2026-04-28 + +CAC predictor — opt-in 4th gating axis for the diagnostic emit +cascade (Cohen's Agentic Conjecture). Introduces a hierarchical +empirical-Bayes scorer over the analytics store that predicts the +probability an agent will adopt the proposed fix for a given +diagnostic, and either suppresses or downgrades emits whose +predicted adoption falls below a configured threshold. **Disabled +by default**; behavior is bit-identical to 0.7.1 until an operator +explicitly enables it from the dashboard. + +### Added — `src/core/cac-config.js` + +Persisted config at `/.pos-supervisor/cac-config.json`. +Mirrors the `rule-overrides.js` pattern (atomic temp+rename writes, +tolerant reads, never throws). Schema: +`{ version: 1, enabled: false, mode: 'shadow' | 'active', threshold: +0.30, action: 'downgrade' | 'suppress', min_samples: 5 }`. +Out-of-range values are coerced to defaults — invalid mode strings, +threshold outside `[0, 1]`, negative `min_samples`, etc. all silently +fall back instead of throwing. Public API: `loadCacConfig`, +`saveCacConfig`, `updateCacConfig`, `defaultCacConfig`, +`VALID_MODES`, `VALID_ACTIONS`. + +### Added — `src/core/cac-predictor.js` + +Pure scoring + decision functions, decoupled from the integration +via dependency injection (`historyProvider` / `severityProvider`): + +- `scoreFixHelpfulness({ rule_id, severity, file_domain, min_samples, + historyProvider, severityProvider })` — hierarchical + empirical-Bayes scorer. Tries `(rule_id, file_domain)` first, falls + back to `(rule_id)` alone, then `(severity)`, then a `Beta(2, 2)` + prior. Returns `{ p_adopted, p_lower, p_upper, n_samples, adopted, + feature, model }` where `feature ∈ { 'rule_id+domain', 'rule_id', + 'severity', 'prior' }`. Re-uses `betaPosterior(...)` already + exported from `analytics-queries.js`. +- `decideAction(prediction, config)` — returns `{ decision, reason }` + where decision is `'allow' | 'downgrade' | 'suppress'`. The + `feature: 'prior'` case (no signal) always allows — the predictor + refuses to gate when flying blind. +- `applyCac(result, { config, analyticsStore, filePath, sessionBus, + log })` — the gate function. Walks `result.errors / warnings / + infos`, scores each diagnostic, and either passes through (shadow + mode) or mutates the result (active mode). Severity downgrades + trigger a bucket rebalance so `result.errors → result.warnings` + reflects the new severity. NEVER throws — predictor / store + failures degrade open. **Predictor only ever suppresses or + downgrades; never adds, never mutates fix proposals.** +- `buildHistoryProvider(analyticsStore)` / + `buildSeverityProvider(analyticsStore)` — real implementations + that issue correlated SQL subqueries against + `diagnostics × outcomes` and return `{ adopted, total }`. Each + provider is wrapped in `safeProvide` so a failed query is treated + as zero samples (falls through the hierarchy). +- In-memory ring buffer of the last 200 decisions + (`getRecentCacDecisions(limit)`) plus a + `sessionBus.emit('cac_decision', ...)` event per decision for the + dashboard's recent-decisions panel. + +### Added — validate-code integration (`src/tools/validate-code.js`) + +New step 12c, inserted between the existing force-disable filter +(step 12b) and the null-hint strip (step 12). Reads +`ctx.cacConfigState?.current.enabled` — when `false`, the call site +short-circuits and validate-code is bit-identical to 0.7.1. When +enabled, `applyCac(...)` is called inside a try/catch — any +predictor failure is logged and diagnostics pass through unchanged. +Skipped when `ctx.untracked` is set (dashboard live-console calls). + +### Added — server wiring (`src/server.js`) + +Shared mutable config ref: +`const cacConfigState = { current: loadCacConfig(projectDir, { log }) +}`. Threaded through `ctx` so validate-code reads the latest config +on every call. New `syncCacConfig()` callback passed to `startHttp` +as `onCacConfigChanged` — POST to `/api/cac/config` triggers it, +re-reading the file and refreshing the live ref without restart +(mirrors the existing `onOverridesChanged` hot-reload pattern for +rule overrides). + +### Added — HTTP endpoints (`src/http-server.js`) + +- **`GET /api/cac/config`** — returns + `{ config, defaults, valid_modes, valid_actions }` for dashboard + bootstrapping. +- **`POST /api/cac/config`** — body is any subset of + `{ enabled, mode, threshold, action, min_samples }`. Unknown keys + are silently dropped; out-of-range values are coerced. Triggers + `onCacConfigChanged` for live-ref refresh. Returns `{ config }` + with the persisted state. +- **`GET /api/cac/decisions?limit=N`** — returns + `{ count, decisions, summary }` from the ring buffer. `summary` + groups by `decision` (allow / downgrade / suppress), `feature`, + and `mode` for at-a-glance dashboard stats. + +### Added — dashboard CAC Predictor panel (`src/dashboard.js`) + +New panel inside the Engine Map tab, sited next to "Adaptive Mode +Impact": + +- **Status badge** (OFF / SHADOW / ACTIVE) with color-coded fill. +- **Three-state toggle** — Off / Shadow / Active. Active requires + `confirm()` (prevents accidental enable). Each click POSTs the + matching patch and re-renders. +- **Threshold slider** (0–1, step 0.05) with live label. +- **Action selector** (Downgrade / Suppress). +- **min_samples** numeric input. +- **Recent decisions** mini-table — last 30 entries with rule_id, + file (last two segments), feature, P(adopted), N samples, + decision, mode. Color-coded decision column. +- Refresh button + auto-fetch when the Engine Map tab is opened. + +CSS additions (`.cac-*` classes) follow the existing AMI / em-panel +style. Browser-side dashboard JS verified via inline `Function()` +constructor parse — passes. + +### Tests + +- `tests/unit/cac-config.test.js` — 13 cases: defaults, missing-file + load, malformed-JSON tolerance, round-trip, invalid mode coerced, + out-of-range threshold clamped, negative `min_samples` rejected, + patch via `updateCacConfig`, unknown-keys dropped. +- `tests/unit/cac-predictor.test.js` — 19 cases covering the scorer + hierarchy (`rule_id+domain` → `rule_id` → `severity` → `prior`), + the decision function (prior always allows; threshold gating with + both `suppress` and `downgrade` actions), the gate (disabled → + no-op, shadow → records-only, active → mutates result, severity + downgrade rebalances buckets, predictor failure passes through, + `.unmatched` synthesized when rule_id is missing, + file_domain derived from `filePath`, ring buffer caps at 200, + `sessionBus.emit('cac_decision', ...)` fires). +- `tests/integration/cac/toggle.test.js` — 8 end-to-end cases + exercising the full HTTP + validate-code path: defaults at boot, + disabled is a true no-op, shadow records but doesn't modify, + active mode is wired without crashing, disabling resets behavior + immediately, garbage POST returns 400, unknown keys dropped, + out-of-range threshold coerced. + +40 new tests (32 unit + 8 integration), all green. Pre-existing +flakes in `tests/integration/scenarios/` and the `0.7.0`-documented +`load_development_guide` drift are unchanged by this release — +verified by stashing the diff and re-running on a clean tree. + +### Safety contract + +The CAC layer is fully separable. Disabling it (`enabled: false` in +config — the default) makes validate-code execute the same code path +as 0.7.1: the integration call site is gated by a single `if +(cacConfig?.enabled && !ctx.untracked)` check. Even when enabled, +the predictor only ever suppresses or downgrades — it never adds +diagnostics, never mutates fix proposals, and never throws (every +boundary is wrapped). Schema migrations are not required — the +analytics DB is unchanged. + +### Fixed — CAC decisions are now persistent + +Two compounding silent failures were dropping every CAC decision on +the floor before reaching disk, leaving the dashboard's "Recent CAC +Decisions" panel empty after every server restart even though the +predictor was firing correctly. + +1. **Missing event-kind registration.** `recordDecision` called + `sessionBus.emit('cac_decision', …)`, but `cac_decision` was + absent from `KIND_SCHEMAS` in `src/core/session-events.js`. + `makeEvent` threw `unknown kind "cac_decision"`, the throw was + swallowed by the `try { sessionBus.emit(…) } catch {}` wrapper, + and the event never reached the NDJSON writer. +2. **Envelope-key collision in the payload.** Even after registering + the schema, the in-memory ring entry carried its own `ts` field + that collided with `ENVELOPE_KEYS` in `makeEvent`, so the next + gate would have thrown `reserved envelope key "ts"` and been + swallowed too. + +Both fixes are in this release: + +- `src/core/session-events.js` — added `CacDecisionPayload` (typed + enums for `feature` / `decision` / `mode` / `severity`, nullable + probability fields for the no-signal `prior` case), registered as + `cac_decision` in `KIND_SCHEMAS`. Pinned by 6 new tests covering + happy path, the `prior` shape with null probabilities, the `ts` + envelope-collision regression, an unknown-decision rejection, and + full NDJSON roundtrip. +- `src/core/cac-predictor.js::recordDecision` — compute `ts` once, + pass it as the `emit(kind, payload, ts)` third argument, strip + `ts` from the payload. Refactored ring push into `pushRingEntry` + shared by live emits and the rehydrator. Added defensive `?? null` + on optional payload fields so `.nullable()` schema constraints + hold even for malformed callers. Bus-failure regression test + asserts a thrown emit no longer drops the in-memory ring entry. + +### Added — CAC decision rehydration on startup + +The 200-entry `recentDecisions` ring lives in module-level memory and +was previously never read from disk on boot, so the dashboard panel +started empty even when prior sessions' NDJSON logs contained +hundreds of decisions. New layer: + +- `loadRecentCacDecisions(sessionsDir, limit)` — pure function. Lists + `/session-*` subdirectories newest-first (session ids + are ISO timestamps, so lexical sort matches chronological), peeks + each line via cheap `JSON.parse` for `kind === 'cac_decision'` + before paying the full `readEvent` Zod cost, sorts the surviving + entries by `ts` ascending, trims to `limit`. Tolerates corrupt + JSON, malformed payloads, future-version events, missing files, + and an absent sessions directory — every error path returns `[]` + so a broken log can never block server boot. Overscan caps I/O at + `2 × limit` candidates across recent sessions. +- `rehydrateRecentCacDecisions(sessionsDir, limit)` — replaces the + ring contents and returns the count. Idempotent; safe to call + before any live emits. +- `src/server.js` — wired immediately after `syncCacConfig()` (uses + the existing `sessionsDir` declared above, no new globals). Logs + `cac-predictor: rehydrated N decision(s) from prior sessions` only + when N > 0; runs even when the predictor is disabled so flipping + it on later in the session doesn't show an empty audit trail. + Try/catch wrapped — boot continues unconditionally on I/O failure. + +13 new tests cover missing dir, empty dir, single-session reads, +mixed-kind sessions, corrupt JSON / partial events / future-version +lines, multi-session chronological merge, limit clamp + most-recent +semantics, idempotence, and ring clearing when the sessions dir is +empty. + +End-to-end verified on a real project: validate_code on a broken +file produced 2 errors → 2 `cac_decision` lines persisted to +`events.ndjson` → server restart → log line `rehydrated 2 +decision(s) from prior sessions` → `/api/cac/decisions` returned +both entries with their original session timestamps preserved. + +### Fixed — `function`/`graphql` tag `lib/` prefix is invalid, never optional + +platformOS resolves `function` tag paths under the partial search +paths declared by `@platformos/platformos-common`: +`FILE_TYPE_DIRS[Partial] = ['views/partials', 'lib']` joined under +`app/`. So `'commands/X'` resolves to `app/lib/commands/X.liquid`, +and `'lib/commands/X'` resolves to `app/lib/lib/commands/X.liquid` +— a directory that never exists in any sane project. The literal +`lib/` prefix is **invalid**, not optional. The `graphql` tag uses +a different search path (`['graphql', 'graph_queries']` under +`app/`), so `'lib/queries/X'` in a `{% graphql %}` tag is doubly +wrong. + +Pos-supervisor was systematically encoding the wrong assumption in +five places — and worse, the fix-generator and rule engine were +**suppressing the LSP's correct `MissingPartial` diagnostic** by +stripping the `lib/` prefix before the disk check, so the agent +saw "no problem" while platformOS would 500 at runtime. Compounded +by ~25 documentation files (hints, references, knowledge.json, +domain-gotchas) that listed `lib/commands/` as the canonical call +form, training every agent reading those docs to write broken code. + +#### Code fixes + +- `src/core/diagnostic-pipeline.js::resolveMissingPartialPaths` — + removed the `name.replace(/^lib\//, '')` call. Now mirrors the + upstream `DocumentsLocator` exactly, returning candidate paths + under `app/views/partials/` and `app/lib/` verbatim. The LSP's + `MissingPartial` for `lib/commands/X` is no longer suppressed. +- `src/tools/analyze-project.js` — same `replace(/^lib\//, '')` + removed from the function-call resolver. `app/lib/${fc.path}.liquid` + is now constructed directly, so `'lib/commands/X'` correctly + resolves to `app/lib/lib/commands/X.liquid` in the error message + and surfaces the bug to the agent. Also extended the iteration to + `commands` / `queries` / `layouts` `function_calls` (previously + only `pages` and `partials` were checked, so a wrong call inside + a multi-phase command's orchestrator slipped through unchecked). +- `src/core/rules/queries.js::classifyPath` — returns + `{ type: 'invalid_lib_prefix', path: null, correctedName }` for + `lib/commands/` / `lib/queries/` instead of stripping. Existing + rules already gate on `path` truthiness, so `file_exists` / + `suggest_nearest` / `create_file` correctly skip these. +- `src/core/rules/MissingPartial.js` — added rule + `MissingPartial.invalid_lib_prefix` at priority 5 (beats every + other branch). Emits a `text_edit` fix using the LSP positions + to swap the quoted reference for its `lib/`-stripped form, with + a guidance fallback when position fields are missing. +- `src/core/fix-generator.js::fixMissingPartial` — handles the + invalid-prefix case before any other branch; emits a `text_edit` + with original quote-style preserved (`'` or `"`, peeked from the + source buffer at the diagnostic column). No longer proposes + creating a phantom file at `app/lib/lib/...`. +- `src/core/error-enricher.js::detectObjectType` / + `buildCreatePath` — recognize `invalid_lib_prefix` as its own + type and route the hint renderer to the new variant template + with the corrected disk path. + +#### New hint variant + +`src/data/hints/MissingPartial-invalid_lib_prefix.md` — explains the +upstream resolver semantics and prescribes "drop the prefix" +instead of the generic "create the file" template. Renders with +both the wrong call form (so the agent recognizes their input) and +the corrected one, and the disk path the corrected call would +resolve to. + +#### Data sweep — ~25 documentation files + +Every `function`-tag use of `'lib/commands/X'` and `'lib/queries/X'` +in `src/data/` rewritten to `'commands/X'` / `'queries/X'`. Every +`graphql`-tag use of `'lib/queries/X'` rewritten to `'X'` (graphql +search path is `app/graphql/`, not `app/lib/queries/`). Touched +files include `knowledge.json`, `domain-gotchas.yml`, +`checks/MissingPartial.yml`, all `references/{partials, pages, +commands, authentication, graphql, liquid, modules, forms}/*.md`, +`domains/{commands, queries}.md`. Three teaching-context references +that explicitly cite `lib/commands/X` as the wrong form +(`Do NOT prepend lib/...`) were preserved deliberately. All YAML / +JSON files re-validated after the sweep. + +#### Tests + +- `tests/integration/analyze-project-lib-prefix.integration.test.js` + — fully rewritten. The previous version pinned the inverse + contract (asserting `lib/commands/X` was NOT flagged when the + bare-form file existed); the rewrite pins the correct one + (`lib/commands/X` MUST be flagged with the doubled `app/lib/lib/` + resolution string in the error message; the bare `commands/X` + form is not flagged). +- `tests/unit/rules/queries.test.js` — `classifyPath` now pinned + on the new `invalid_lib_prefix` shape with `correctedName`. +- `tests/unit/rules/MissingPartial.test.js` — 7 new tests for the + `invalid_lib_prefix` rule (text_edit happy path, guidance + fallback when positions are missing, `lib/queries/` symmetry, + doesn't fire for bare `commands/X`, doesn't fire for module + paths, beats `create_file` even when the corrected file doesn't + exist on disk). +- `tests/unit/diagnostic-pipeline.test.js` — 4 new tests for + `verifyMissingPartialsOnDisk` (suppresses bare-form cache lag, + does NOT suppress `lib/`-prefixed errors even when the bare-form + file exists on disk, symmetric for queries, still suppresses + legitimate non-`lib/` cache-lag misses). +- `tests/unit/error-enricher.test.js` — 2 existing tests rewritten + to use canonical syntax; 1 new regression test pinning that the + invalid-prefix variant fires "drop the prefix" copy and never + the create-file template, with the single-`lib/` corrected disk + path always in the hint. + +Targeted: 373/373 pass across 22 touched test files. Full suite: +2238/2243 pass — same 5 pre-existing failures from main (CRUD +scenario timeout cascade and `load_development_guide` MANDATORY +WORKFLOW), zero new regressions. + +The trigger for this work was a session report on 2026-04-29: an +agent failed repeatedly to call commands from a page, concluded +that path resolution was caller-relative ("two valid styles that +look the same but behave differently"). The diagnosis was wrong +(resolution is global, not caller-relative), but the symptom was +real and ours — agents kept writing `lib/commands/X` because our +docs said to, and our suppression hid the LSP's correct rejection. + +### Fixed — Commands domain references contradicted modules/core docs + +The `references/modules/core/*.md` docs were modernized for +pos-cli 6.0.7+ (canonical syntax, app-level build/check phases, +validators at `modules/core/lib/validations/`), but the +parallel `references/commands/*.md` docs still showed the **legacy** +API: phantom `modules/core/commands/build` and `modules/core/commands/check` +helpers, an array-of-validators shape (`validators: [{...}]`) +passed to a single check helper, validators called at the wrong +path (`modules/core/validations/` instead of +`modules/core/lib/validations/`), and validator argument order +diverging from the actual `@param` order. + +Net effect: `domain_guide(commands, patterns)` returned a fake API +that would 500 at runtime, while `module_info(core, patterns)` +returned the correct one. Agents got opposite advice from the two +tools depending on which they consulted first. A real session +report on 2026-04-29 documented an agent following the wrong +domain_guide and producing a non-working command file. + +#### Authoritative pattern (now consistent across both tools) + +Three files per command action: orchestrator + sibling +`/build.liquid` + sibling `/check.liquid`. Only +`modules/core/commands/execute` is module-level. Validators chain +individually with `modules/core/lib/validations/` and argument +order `c, field_name, object, [options...]`. + +```liquid +{% function object = 'commands/products/create/build', object: params %} +{% function object = 'commands/products/create/check', object: object %} +{% function c = 'modules/core/lib/validations/presence', + c: c, field_name: 'title', object: object %} +{% function object = 'modules/core/commands/execute', + mutation_name: 'products/create', + selection: 'record_create', + object: object %} +``` + +#### Files rewritten — `src/data/references/commands/` + +- `README.md` — minimal orchestrator example now uses the canonical + three-file pattern. Removed every legacy build/check reference + that wasn't an explicit anti-pattern callout. +- `configuration.md` — directory tree shows + `.liquid` + `/build.liquid` + `/check.liquid` + per CRUD operation. Naming-conventions table includes the new + "Phase call" row. Command file template rewritten as the + three-file canonical layout. +- `api.md` — fully rewritten. Removed the phantom + `modules/core/commands/build` and `modules/core/commands/check` + sections. Validator family now keyed at + `modules/core/lib/validations/` with the modern names + (`number`, `matches`, `equal`, `included`, …). New "Legacy + Forms — No Longer Supported" appendix lists every renamed + validator and the `validators: [...]` shape so existing agents + reading legacy docs know what to migrate. +- `patterns.md` — fully rewritten. CRUD examples (create / update / + delete / event-publishing / conditional validation / error + display / command composition) all use the canonical + three-file shape with chained `lib/validations/` calls. +- `gotchas.md` — TOP GOTCHA section explicitly framing the + phantom `modules/core/commands/build` / `…/check` as the most + common error. New entries for the wrong validator path + (`modules/core/validations/` vs `modules/core/lib/validations/`), + the legacy `validators: validators` array shape, and the new + argument order. Troubleshooting flowchart updated. +- `advanced.md` — transactions / composition / custom validation / + uniqueness / file uploads / idempotent / debugging — all + rewritten to the canonical three-file shape. The transaction + example no longer reaches for a phantom + `modules/core/commands/build` for line items. + +#### Secondary doc sweep + +- `references/partials/patterns.md` — Command Partial Pattern + example rewritten to use `commands/products/create/build` and + `commands/products/create/check` (was using the phantom helpers). +- `resources/ok-platformos-development-guide.md` and + `resources/short-platformos-development-guide.md` — Check Stage + examples updated: + `modules/core/validations/presence` → + `modules/core/lib/validations/presence`, with the canonical + argument order (`c, field_name, object`). The "DEPRECATED — DO + NOT USE" anti-pattern callout was already correct and was left + intact. +- `knowledge.json` and `modules-missing-docs.json` — entry + `modules/core/validations/presence` (used by the + MetadataParamsCheck false-positive suppression list) corrected + to `modules/core/lib/validations/presence`. JSON files + re-validated. + +#### What was deliberately NOT changed + +Every reference to `modules/core/commands/build` / +`modules/core/commands/check` / `modules/core/validations/` that +remains in the docs is now an **anti-pattern teaching reference**: +either inside a "DO NOT", "✗ WRONG", "Template not found", +"Legacy shape", or `TOP GOTCHA` block. Removing those would lose +the authoritative "this path doesn't exist; here's why" copy +agents need when they hit the error in the wild. + +The authoritative `references/modules/core/*.md` docs were +already correct and remain unchanged. The `commands` domain now +mirrors them. + +## 0.7.1 — 2026-04-28 + +Fix for the `GraphQLVariablesCheck.required` regression spiral +reported on 2026-04-27 (4 emits / 100 % regression on +`app/lib/commands/contacts/create.liquid` in DEMO) and the dashboard +404 on rule-driven check drilldowns. + +### Fixed — `GraphQLVariablesCheck.required` parser blind spot + +Root cause: a `{% graphql %}` call written inside a `{% liquid %}` +block with multi-line `,` continuation. Both `liquid-html-parser` and +pos-cli's LSP truncate the call at the first newline-comma — +`markup.args` ends up empty and LSP fires +`GraphQLVariablesCheck.required` for every named arg past it. The +agent sees the args in source, our `.required` hint says "add the +variable", agent rewrites cosmetically, LSP fires the same errors. +Loop. + +Three coordinated changes resolve the spiral: + +- **`liquid-parser.js` — graphql call extraction enriched.** Each + `extracted.graphql` entry now carries `args: [name, …]` (from + `markup.args`) and `source_kind: 'tag' | 'liquid_inline' | + 'liquid_multiline_truncated'`. Truncation is detected when the + call's source range starts without `{%` (we are inside `{% liquid + %}`), ends on `,`, and the file text immediately past the call has + another `name:` clause on a subsequent line. New + `classifyGraphqlSourceKind` exported for reuse. Dedup-by-queryName + preserved; if any duplicate call is truncated, the existing entry's + `source_kind` upgrades to the most-pessimistic value so downstream + rules can detect the symptom regardless of which call won the dedup. +- **`rules/GraphQLVariablesCheck.js` — new `parser_blind_spot` + sub-rule (priority 3, before `.required` at 5, confidence 0.95).** + Fires when `direction === 'required'` AND the project graph reports + any graphql call in the file with `source_kind === + 'liquid_multiline_truncated'`. Hint redirects the agent at the + syntactic root cause: convert to single-line `{% graphql … %}` tag + form, or keep it inside `{% liquid %}` but place every named arg on + the same line as `graphql`. Falls through to `.required` when the + call is fine — purely additive, no risk of misfire on legitimate + missing-variable diagnostics. +- **`structural-warnings.js` — new + `pos-supervisor:GraphqlMultilineInLiquidBlock` (severity: error).** + Surfaces the syntactic cause once per truncated call, before LSP + enrichment runs. Reuses `classifyGraphqlSourceKind`. Fires for all + domains; partials still get the existing `GraphqlInPartial` error + on top. + +### Fixed — Dashboard hint endpoint 404 on rule-driven checks + +The dashboard's rule drilldown panel `GET +/api/hints?name=` 404'd for the 12+ rule-driven checks that +have no `src/data/hints/.md` file (`GraphQLVariablesCheck`, +`PartialCallArguments`, `MissingRenderPartialArguments`, +`UnusedDocParam`, `LiquidHTMLSyntaxError`, +`pos-supervisor:InvalidLayout`, `pos-supervisor:MissingSlug`, +`pos-supervisor:MissingContentForLayout`, +`pos-supervisor:SchemaProperty`, `pos-supervisor:SchemaYAML`, +`pos-supervisor:DeprecatedTag`, +`pos-supervisor:TranslationMissingLocaleKey`). These checks are +served by `src/core/rules/.js` modules, not static markdown — but +the endpoint was hardwired to `readFileSync('.md')`. + +- **`http-server.js` — `handleGetHints` now branches.** md file + present → returns `{ source: 'static', content }`. md missing but + rule registry has the check → returns `{ source: 'rule', content, + rule_ids }` with a synthesized markdown reference (per sub-rule: + `id`, priority, truncated `when()` source, footer pointing at + `src/core/rules/.js`). Both miss → 404. List endpoint unions md + filenames with `getAllChecksWithRules()` and adds a `checks: [{ + name, sources: ['static'|'rule', …] }]` companion field. Backward + compat preserved on the `hints` array. +- **`dashboard.js` — drilldown is source-aware.** The hint panel + title surfaces `src/core/rules/.js` for rule-driven checks + instead of the misleading `(src/data/hints/.md)` it always + showed. Action recommendations ("edit X to rewrite the hint") + point at whichever file actually owns the hint. + Knowledge-browser `loadHint` renders a `[RULE-DRIVEN]` / + `[STATIC]` source badge above the body and a readable `404` + message instead of an empty `
`. Strips `pos-supervisor:`
+  prefix from the rule module path — the rule files are not
+  namespaced.
+
+### Tests
+
+- `tests/unit/liquid-parser.test.js` — 7 new cases covering `args` +
+  `source_kind` for tag, `liquid_inline`,
+  `liquid_multiline_truncated` forms; the
+  comma-ending-without-trailing guard; and the dedup-upgrade path.
+- `tests/unit/rules/Tier3RulesPhase3.test.js` — 5 new cases covering
+  `parser_blind_spot` priority, fall-through to `.required` when not
+  truncated, fall-through when file is unindexed, and fall-through
+  when no graph is available.
+- `tests/unit/structural-warnings.test.js` — 5 new cases covering
+  the `GraphqlMultilineInLiquidBlock` detector against truncated,
+  tag-form, single-line liquid-block, multi-line tag-form, and
+  multiple-occurrence inputs.
+- `tests/unit/http-server.test.js` — 6 new cases covering list
+  union, static md retrieval, rule synthesis, unknown-check 404,
+  prefixed (`pos-supervisor:…`) rule round-trip, and the
+  static-wins-over-rule precedence.
+
+23 new tests, all green. Browser-side dashboard JS verified via
+inline `Function()`-constructor parse.
+
+## 0.7.0 — 2026-04-27
+
+Rule-engine and hint quality overhaul driven by the
+2026-04-27 DEMO performance report (`docs/rule-performance-plan.md`):
+123 diagnostics across 17 sessions, fix-proposal rate 24 %, six rules
+flagged AT RISK or HARMFUL. The headline shift after this release: every
+high-volume bucket-B `.unmatched` check now lands with a stable rule_id
++ structured guidance fix; AT RISK rules ship locale-aware /
+intent-aware hints that converge on the canonical platformOS shape
+instead of producing contradictory advice.
+
+### Fixed — AT RISK rules (Bucket A)
+
+- **`MissingPartial.module_path` (was 0 % resolve / 100 % regress).** Hint
+  diagnosed the symptom but never named a target. Rule now enumerates
+  available module call paths from the filesystem at apply time, runs
+  Levenshtein over them, and ships the top-5 candidates inline. Special
+  case for `modules//commands/build` and `…/check` — the hint
+  explicitly explains build / check are inline phases of the agent's
+  own command and only `execute` is exported by core. Rewrote
+  `MissingPartial.md` STEP 1 to remove the misleading "use core's
+  execute helper" example that drove the over-generalisation. New
+  `src/core/rules/module-paths.js` helper (sync filesystem walk
+  mirroring `module-scanner.scanPublicApi`); `projectDir` plumbed
+  through `enrichCtx` → facts in `validate-code.js` and
+  `error-enricher.js` (both `enrichError` and
+  `bridgeRulesOntoUnattributed`).
+- **`TranslationKeyExists.suggest_nearest` (was 0 % resolve, 6 / 6 ignored).**
+  Three distinct failure modes uncovered, all fixed:
+  - **Locale-prefix double-up.** `flattenYaml` over a properly-rooted
+    `en.yml` produces keys like `en.app.user.title` (the YAML root IS
+    the locale). The rule was suggesting `en.app.user.title` for an
+    agent's `app.usr.title` typo; agents wrote `'en.app.user.title' | t`
+    which Liquid resolved to `en.en.app.user.title` — adopting the fix
+    re-broke the lookup. `translationKeysForLocale` now strips leading
+    `.`; `array_index_misuse` and `create_key` strip from the
+    agent's key before computing `arrayKey` / YAML snippets;
+    `suggest_nearest` matches against bare AND prefixed forms, picks
+    closer. New `stripLocalePrefix(key, locale)` exported from
+    `queries.js`. Hint and fix descriptions now explicitly warn against
+    including the `en.` prefix.
+  - **Levenshtein threshold too loose for dotted keys.** Shared helper's
+    `length * 0.6` admitted distance-10 matches on 20-char keys. Local
+    bound `min(5, length / 3)` per call site — brand-new keys fall
+    through to `create_key` instead of attracting bogus "did you mean"s.
+  - **Defensive `[N]` gate.** Every subrule (`array_index_misuse`,
+    `suggest_nearest`, `create_key`) now gates on raw `diag.message` in
+    addition to `params.key`. Belt-and-suspenders against extractor drift.
+- **`pos-supervisor:NonGetRenderingPage` (was 20 % resolve / 20 %
+  regress, 25 outcomes).** Per the
+  `docs/rule-performance-plan.md` gist analysis: split into three
+  intent-aware subrules + defensive default. `validatePageMethodAndForms`
+  in `structural-warnings.js` (renamed from
+  `validateNonGetRenderingPage`) emits discriminator-prefixed messages;
+  the rule layer routes by regex.
+  - `api_renders_html` — slug under `/api/`, `/_/`, `/internal/` + non-GET
+    method + (HTML present OR `format: json` missing). Hint ships the
+    canonical `format: json` + GraphQL JSON body shape.
+  - `html_on_post` — non-API slug + non-GET method + HTML rendering. Hint
+    disambiguates "landing page" vs "API handler" intents with
+    copy-pasteable Liquid for both.
+  - `get_form_target` — GET page hosts `
` + where X isn't under API prefixes and isn't the page's own slug. Hint + routes the agent to `/api/` + auto-creates the API page path. + - Form parsing: attribute-order-independent regex; supports single, + double, and unquoted attribute values; self-post detection + (`action == own slug`) prevents false positives for sanctioned + self-post pages. API page emit policy: only layout / partials / + HTML tags count as HTML rendering — bare `{{ … }}` in `format: json` + pages is the intended JSON serialization, NOT flagged. +- **`pos-supervisor:InvalidLayout` and `ValidFrontmatter.layout_missing` + duplicate-emit + wrong path.** Two checks fired on the same root + cause with diverging line numbers (line-only dedup let both through), + and the structural emitter hardcoded `.html.liquid` for the create_file + proposal — DEMO uses `.liquid`, so agents accepted the fix and the + file landed at the wrong path. + - `validateLayout` in `structural-warnings.js` now calls + `detectLayoutExtension(projectDir, moduleName)` to sample existing + layouts and pick `.liquid` vs `.html.liquid` (defaults to `.liquid` + when the layouts dir is empty — modern convention). + - `extractLayoutPath` in `fix-generator.js` lifts the path verbatim + from the message's `Expected file: \`...\`` clause — single source + of truth, no per-file re-derivation. + - `suppressUpstreamFrontmatterDup` now matches by **layout name** in + addition to line — same root cause regardless of line drift. + - New `src/core/rules/InvalidLayout.js` rule attaches stable rule_id + + matching create_file fix at the corrected path. + +### Added — Bucket B `.unmatched` promotions (rule modules) + +13 new rule modules covering every bucket-B `.unmatched` check from the +performance report. Each ships stable rule_id + structured guidance; +where a heuristic text_edit already exists in `fix-generator.js`, the +rule emits `guidance` only and the heuristic stays as the actionable +diff. End-to-end attribution verified via `bridgeRulesOntoUnattributed`. + +- **`DeprecatedTag` (covers both upstream LSP and `pos-supervisor:DeprecatedTag`).** + Subrules `include` (route to `{% render %}` w/ isolated-scope + caveat), `hash_assign` (`{% assign x["k"] = v %}`), `parse_json` + (`| parse_json` filter form), default. Defensive when-gates check + both `params.tag` and raw message regex (the structural variant has + no extractor). +- **`UnrecognizedRenderPartialArguments`.** Extracts argument + partial + from message; emits 3-option decision tree (drop / declare / rename). + Disables option B for module partials (read-only). +- **`SchemaProperty` (8 sub-IDs)** — routes `pos-supervisor:SchemaProperty` + emits by regex into `builtin_conflict` / `duplicate_name` / + `invalid_identifier` / `snake_case` / `upload_options` / + `missing_field` / `misleading_key` / `default`. +- **`SchemaYAML`, `MissingSlug`, `MissingContentForLayout`** — + promotions of existing fix-generator heuristics; rule emits guidance, + heuristic still owns the text_edit. +- **`ParserBlockingScript`** — defer / async / end-of-body decision tree. +- **`TranslationMissingLocaleKey`** — extracts locale from message, + emits before/after YAML wrap recipe. +- **`MissingAsset` (3 subrules)** — + `missing_subdir_prefix` (high confidence: bare `logo.png` matches + existing `images/logo.png`) → `suggest_nearest` (Levenshtein) → + `create_file`. Replaces the heuristic's blind-create proposal. +- **`OrphanedPartial`** — emits `delete_file` fix when graph has 0 + callers; softer guidance for layouts; cites `pending_files` + workflow for in-progress refactors. +- **`MissingPage` (2 subrules)** — `typo` (Levenshtein vs graph page + slugs) → `default` (3-option decision tree + create_file at inferred + path). Handles root route correctly (omit `slug:`). +- **`LiquidHTMLSyntaxError` (5 subrules)** — `unknown_tag` (Levenshtein + vs `tagsIndex.platformOSTags()`) → `for_loop_args` → + `missing_assign` → `inline_literal` → `default`. +- **`PartialCallArguments` (4 subrules)** — highest-volume bucket-B + check (28 emits in DEMO). New extractor parses both + `Required parameter X must be passed to (render|function) call` and + `Unknown parameter X passed to ...` shapes. Subrules + `required_render`, `required_function`, `unknown_render`, + `unknown_function` ship copy-pasteable forwarding patterns + the + canonical drop / declare / rename resolution. Cross-references the + sibling `MissingRenderPartialArguments` / + `UnrecognizedRenderPartialArguments` checks (which carry the partial + path when they co-fire). +- **`GraphQLVariablesCheck` (2 subrules + default)** — new extractor + for `Required parameter X must be passed to GraphQL call` / + `Unknown parameter X passed to GraphQL call`. Hint surfaces a + per-call **signature block** when the file's `graphql_calls` are + indexed — lists every operation invoked + its declared + `$var: Type` list parsed from the .graphql operation header. +- **`UnusedDocParam`** — caller-aware confidence: 0.8 when graph shows + zero callers (option B = remove `@param` is safe); 0.65 when callers + exist (removing the declaration becomes a contract change). Hint + references the pipeline's `suppressUnusedDocParams` so agents + understand surviving emits aren't the named-arg false positive. + No text_edit — contract change with cross-file blast radius is not + safe for the rule layer to automate. + +### Added — query helpers + +- `assetNames(graph)` — list every indexed asset path + (`MissingAsset.suggest_nearest`). +- `stripLocalePrefix(key, locale)` — translation-key normalisation. +- `parseModulePath(name)` exported from `MissingPartial.js` — + splits `modules///` for analytics callers. + +### Changed — graph plumbing + +- `project-scanner.js` and `project-fact-graph.js` now propagate + `graphql_calls` to **pages, partials, AND layouts** (previously + commands/queries only). Without this, `GraphQLVariablesCheck`'s + signature block was empty for the most common caller — API pages + emitting JSON. + +### Changed — extractors + +- New `extractParams` entries for `PartialCallArguments`, + `GraphQLVariablesCheck`, `UnusedDocParam` in `diagnostic-record.js`. + +### Tests + +130 new unit tests across 7 new rule-test files +(`module-paths.test.js`, extended `MissingPartial.test.js`, +`TranslationKeyExists.test.js`, `Tier3Rules.test.js`, +`Tier3RulesPhase2.test.js`, `Tier3RulesPhase3.test.js`, +`DeprecatedTag.test.js`, `NonGetRenderingPage.test.js`, +`InvalidLayout.test.js`). Existing +`error-enricher-bridge.test.js`, `Tier1Rules.test.js`, and +`structural-rule-attribution.test.js` updated to match the +three-subrule shape. + +Total rule entries: **86 (vs 47 at 0.6.0)**. Full suite at release: +1802 / 1803 unit pass (the lone failure is a pre-existing +`load_development_guide` drift unrelated to this release); 88 / 88 +targeted integration pass. + +## 0.6.0 — 2026-04-24 + +Analytics pipeline overhaul + neuro-symbolic engine rounds out. Headline numbers on the DEMO project between 2026-04-23 and 2026-04-24: fix-proposal rate rose from effectively 0 (the emit loop was reading the wrong field) to 45 / 99 (45%); classified fix adoption rose from 0 to 31; confidence coverage from 0% to 89% of emits; rule performance table from 3 entries at baseline to 30+; health score from 91 to 95/100. + +### Fixed — three critical analytics bugs + +- **`outcomes.fix_applied` was always null.** The classifier (`classifyFixAdoption`) existed in `window-classifier.js` but no call site. Wired it into `analytics-store.classifyAndStoreWindows()` using `buildEmitIndex` + blobStore content lookup. Start-of-window emit picks the proposed-fix set the agent actually saw; `regressed` / `write_unverified` outcomes skipped (no semantic meaning). `openAnalyticsStore(dbPath, { blobStore })` now accepts the blob store; `server.js` and `scripts/rebuild-analytics.js` pass it through. +- **`hint_md_hash` emitted but dropped on ingest.** No column existed on `diagnostics`. Schema bumped to v5 with `migrate_v4_to_v5` adding the column; ingestion persists the hash; `diagnosticJourney` + `ruleDrilldown` surface it; dashboard code-context panel renders the hint blob alongside the file window. +- **Heuristic fix-generator fixes never reached analytics.** Emit loop read `d.fixes` (rule-engine channel) but the heuristic generator writes to `d.fix` (singular). Unioned both channels; every fix now persisted with its attribution. + +### Added — Phases A1–A4 (analytics integrity) + +- **A1 — outcome dedup.** `outcomes` table carries UNIQUE(session_id, file, fp); `INSERT OR REPLACE` stamps terminal state as `classifySession` walks windows. Migration `migrate_v1_to_v2` dedups existing rows by MAX(id), drops orphans, adds the index. Resolution > Emit mismatch eliminated. +- **A2 — confidence defaults.** `DEFAULT_CONFIDENCE_BY_SEVERITY = { error: 0.9, warning: 0.7, info: 0.5 }` + `STRUCTURAL_DEFAULT_CONFIDENCE = 0.75`. New pipeline step `populateDefaultConfidence` (step 17) fills any missing confidence and stamps `${check}.unmatched` rule_id fallback. Exported as `stampDefaultsOn(result)` for validate-code to re-run after late structural-warning pushes. +- **A3 — `_source: 'dashboard_live'` untracked gate.** Live Diagnostic Console calls no longer pollute analytics. `tools.js` sets `ctx.untracked = true` on a per-call context copy (restore-in-finally); `validate-code.js` gates `sessionBus.emit('validator_emit', ...)` on `!ctx.untracked`. One-off cleanup script for pre-A3 pollution in `scripts/cleanup-live-console-rows.js`. +- **A4 — rule attribution.** `${check}.unmatched` fallback lands in rule_id when no rule fires. `rulePerformance(store, { minEmitted = 1 })` separate reporting-view query, groups on rule_id including `.unmatched`, exposes `source` and `unmatched` flag. `ruleScores()` stays at minEmitted=5 with `.unmatched` excluded for promotion gating. + +### Added — I1 heuristic + rule fix attribution + +- Schema v6 + `proposed_fixes.rule_id` column + `idx_fixes_rule` index + `migrate_v5_to_v6`. +- Central stamp in `fix-generator.js`: every heuristic fix tagged `heuristic:.` in one place (no per-branch boilerplate). +- Emit loop propagates fix-level rule_id with `f.rule_id ?? d.rule_id ?? null` fallback — rule-engine rules attach rule_id to the HintResult rather than each fix, so fixes inherit from the diagnostic. +- New `fixRulePerformance(store, { minProposed = 1 })` query groups on `proposed_fixes.rule_id`, returns `{ rule_id, source, fix_kind, proposed, outcomes, adopted_verbatim, adopted_partial, adoption_rate, resolution_rate }`. +- HTTP endpoint `GET /api/analytics/fix-rule-performance`. + +### Added — Part G adaptive-mode impact panel + +- `adaptiveModeImpact(store, { windowMs = 86400000 })` returns window-scoped emit counts, rule-matched counts, confidence stats, and an `emits_by_rule` map for counterfactual calculation. +- HTTP `GET /api/engine/impact` merges the query with live engine state (`getDisabledRuleDetails`, force-enable/disable sets) and computes `suppressed_by_disabled` counterfactual. +- Dashboard — new "Adaptive Mode Impact" section in the Engine Map tab: summary stat tiles, disabled-rules table with per-row action buttons, force-enabled / force-disabled chip lists. + +### Added — I4 manual rule overrides + +- `src/core/rule-overrides.js` module. Persists `.pos-supervisor/rule-overrides.json` (atomic write via temp + rename, tolerant read). API: `loadOverrides`, `saveOverrides`, `addForceEnable`, `addForceDisable`, `removeOverride`, `overrideSets`. +- Engine — `_forceEnabled` and `_forceDisabled` sets. `ruleIsActive()` precedence: `force_disable > force_enable > _disabledRules`. New `isCheckForceDisabled(checkName)` also gates structural / LSP-only checks by name. +- Validate-code filter step drops diagnostics whose `check` or `rule_id` is in the force-disable set — structural checks like `pos-supervisor:HtmlInPage` can be killed without waiting for the auto-disable threshold. +- HTTP `GET` and `POST /api/engine/rule-overrides` with `{ action, rule_id, reason }` where action is `force_enable | force_disable | clear`. `onOverridesChanged` hook refreshes the engine without restart. +- Dashboard — override-add form (input + reason + FE/FD buttons) with HTML5 `` autocomplete populated from rule-performance data + derived check names. + +### Added — late-push attribution bridge + +- `bridgeRulesOntoUnattributed(result, ctx)` in `error-enricher.js`. Runs `runRules` on any diagnostic whose `rule_id` is still unset and whose `check` has a registered rule module. Copies `rule_id`, `hint_md`, `confidence`, `see_also`, `fixes`, `case_base_signal` onto the diagnostic. Idempotent. Rule failures non-fatal. +- Called from `validate-code.js` after all late-push sources (structural warnings, schema/translation validators, diff-aware checks, new-partial caller check) and before `stampDefaultsOn`. Structural `pos-supervisor:*` rules now get their canonical rule_id instead of landing in `.unmatched`. + +### Added — engine-map write-closed windows + draft detection + +- `classifyWriteWindow(validateCall, writeEvent)` and `extractWriteEvents(events)` in `window-classifier.js`. +- Schema v4 adds `windows.is_draft` and `windows.closed_by ∈ {'validate','write'}`. Validate-to-validate windows with no intervening disk write are tagged `is_draft = 1` (measures thinking, not effectiveness). +- `fs-watcher.js` emits `rel_path` alongside `path` so the classifier can match writes to validated files. + +### Added — Tier 1 rule modules + +- `src/core/rules/ImgLazyLoading.js` (rule_id `ImgLazyLoading.recommended`). +- `src/core/rules/ImgWidthAndHeight.js` (rule_id `ImgWidthAndHeight.recommended`). +- `src/core/rules/ConvertIncludeToRender.js` (rule_id `ConvertIncludeToRender.default`). + +Each provides attribution + an action-oriented hint. Fix text stays with the heuristic generator (single source of truth on AST position math); the rules return `fixes: []` and rely on the `heuristic:.text_edit` channel. Registered via `src/core/rules/index.js`. + +### Added — new structural checks + +- **`pos-supervisor:NonGetRenderingPage`** — warns when a page has `method: post/put/delete/patch` AND renders HTML (layout, partials, `{{ }}` output, or HTML tags present). Catches the agent-confusion pattern of setting `method: post` on landing pages, which makes them 404 on browser GET. Suppressed when slug starts with `/api/`, `/_/`, `/internal/` OR the body has no UI signals (pure JSON/redirect endpoint). Rule module `NonGetRenderingPage.default` + hint file `src/data/hints/pos-supervisor:NonGetRenderingPage.md` with a landing-page vs API-endpoint decision tree. +- **`verifyMissingPartialsOnDisk`** pipeline step — cross-check `MissingPartial` diagnostics against the real filesystem; suppress when the partial exists on disk but the LSP hasn't re-indexed yet (handles scaffold write → re-validate timing race). + +### Changed + +- **`pos-supervisor:HtmlInPage` guard** — suppress when the page renders at least one partial (composite landing-page pattern). Production showed 100% regression on this rule before the guard. +- **`pos-supervisor:MissingDocBlock` scope** — dropped commands branch (production showed 40% regression on `commands/`; many internal helpers legitimately don't need doc blocks). Partials only now. +- **Validate-code emit loop** — propagates `rule_id` on every fix; unions rule + heuristic fixes; re-runs `stampDefaultsOn` after all late-push sources so confidence / rule_id coverage is complete. + +### Added — dashboard features + +- **Code-context panel in rule drilldown**: fetches content blob (`GET /api/blob?hash=…`), fix blob, and hint blob in parallel; renders a 40-line window around `fix_range` with the error line highlighted; Proposed fix + Hint blocks below. New `/api/blob` endpoint with 64-hex SHA256 validation. +- **Journey timeline clickable nodes**: click a session dot to open the same code-context panel inline. +- **Confidence column** in the rule-drilldown samples table, color-coded (≥0.8 green, ≥0.5 yellow, else red; `n/a` muted). +- **Live-console file picker** stays in sync with validation SSE activity (`addToLivePickerFiles` / `removeFromLivePickerFiles`) — no longer requires an Explorer tab refresh to see newly-validated files. + +### Added — scripts + +- **`scripts/rebuild-analytics.js`** — rebuild the analytics DB from session event logs. Injects the blob store so fix-adoption classification runs on replay. +- **`scripts/cleanup-live-console-rows.js`** — one-off purge of pre-A3 `__pos_live_console__` rows from events/diagnostics/outcomes/windows/proposed_fixes. + +### Added — tests + +New unit test files: + +- `tests/unit/error-enricher-bridge.test.js` — 6 cases covering bridge idempotency, no-rule-module no-op, missing fact-graph no-op, errors/warnings/infos, rule-throws non-fatal. +- `tests/unit/rule-overrides.test.js` — 7 cases: round-trip, malformed JSON, mutual exclusion. +- `tests/unit/rule-engine-overrides.test.js` — 7 cases: force precedence, check-name gating, engine-state reset. +- `tests/unit/rules/Tier1Rules.test.js` — 4 rule-module tests (ImgLazy, ImgW&H, ConvertInclude, NonGetRenderingPage). + +Extended unit files: `analytics-store.test.js`, `analytics-queries.test.js`, `analytics-queries-k.test.js`, `diagnostic-pipeline.test.js`, `structural-warnings.test.js`, `window-classifier.test.js`, `case-base.test.js`. + +New integration tests in `tests/integration/analytics/`: + +- `untracked.test.js` — A3 gate. +- `fix-rule-attribution.test.js` — I1 follow-up rule-engine inheritance. +- `force-disable-check.test.js` — I4 override semantics end-to-end (POST + clear). +- `structural-rule-attribution.test.js` — bridge end-to-end (NonGetRenderingPage lands as `.default`, not `.unmatched`). + +**Suite totals: 1635 unit + 25 analytics/http/workflows integration, all green.** + +### Changed — plan doc + +- `docs/new-task/implementation-plan.md` — new "Addendum — 2026-04-23" section: I1 (heuristic rule attribution), I2 (see_also_followed outcome), I3 (soak fresh data), I4 (manual rule re-enable + dashboard visibility). Revised short-term order with Part G + I4 bumped up. + +### Migrations + +DB schema: **v1 → v6** via five numbered, idempotent steps. No backfills write data — only reshape tables. A `store.rebuild(sessionsDir)` against the existing event log repopulates the new columns. + +- **v1 → v2**: dedup outcomes + add UNIQUE(session, file, fp) index + add `session_id` / `file` columns + backfill from windows. +- **v2 → v3**: dedup diagnostics + add UNIQUE(session, file, fp) index. +- **v3 → v4**: add `windows.is_draft` + `windows.closed_by`. +- **v4 → v5**: add `diagnostics.hint_md_hash`. +- **v5 → v6**: add `proposed_fixes.rule_id` + `idx_fixes_rule`. + +### Upgrade notes + +1. `pkill -f bin/pos-supervisor.js && bun bin/pos-supervisor.js` — new schema migrations run on first open. +2. Optional: `bun scripts/rebuild-analytics.js /path/to/project` — replays the event log into the new columns so historical sessions gain confidence / hint_md_hash / fix rule_id attribution. +3. Optional: `bun scripts/cleanup-live-console-rows.js /path/to/project` — purge pre-A3 live-console pollution if the DB predates this release. + ## 0.5.2 ### Added diff --git a/SYSTEM_ARCHITECTURE.md b/SYSTEM_ARCHITECTURE.md new file mode 100644 index 0000000..fd0c2e2 --- /dev/null +++ b/SYSTEM_ARCHITECTURE.md @@ -0,0 +1,945 @@ +# pos-supervisor — system architecture + +A walkthrough of how the validator turns a `validate_code` call into a +diagnostic with hints and proposed fixes, what every data file under +`src/data/` actually does, what the dashboard's vocabulary +(`unmatched`, `active`, `adoption`, `collateral`) actually measures, and +how the adaptive engine and CAC predictor read analytics back into the +emit path. + +The document is meant to make the system legible end-to-end: after +reading it you should be able to look at a row in the dashboard and +trace it backwards to a concrete file you can edit. + +--- + +## 0. Reading guide for the report you generated + +The numbers from `pos-supervisor-report-2026-05-01_18-31-50.md` line up +with the concepts below, so a quick orientation: + +- **Funnel: 357 emitted → 254 resolved (71%), 23 regressed (6%).** + 254/357 windows were resolved across 71 sessions. That's "we said + something useful 7 times out of 10". 6% regression rate is low but + not zero — the agents took our fix and broke something else in 23 + cases. Anything tagged `HARMFUL` in the rule table contributed to + those 23. +- **Health score 15/100 (infrastructure only).** That's not "the + validator is broken" — it's "the dashboard's project-analysis tab was + never run on this DEMO project, so the project-shape dimensions stay + zero". Click "Analyze Project" once and the score takes its real + value. +- **`PartialCallArguments`: 80 emits, 87% resolved, 10% regressed, + GOOD.** This is the workhorse — it fires the most and the agent + almost always gets it right. The fact that + `PartialCallArguments.unmatched` accounts for 49 of those 80 is the + interesting part: the rule engine doesn't have a specific + rule_id for ~60% of these emits, just the catch-all (see §4.4 below). + Adding a few `PartialCallArguments.` rules would be a real + effectiveness win. +- **`MissingPage`: 14 emits, 25% resolved, LOW.** Most of those + 14 are the self-page false positive we just fixed (Issue 4). + Resolution rate should climb on the next run. +- **`NonGetRenderingPage.get_form_target`: 1 emit, 100% regressed, + -100% effectiveness, INSUFFICIENT_DATA.** Don't act on this row yet — + one regression on one emit is not enough signal. The + `INSUFFICIENT_DATA` label is doing exactly what it should: blocking + panic. +- **`UNMATCHED` rule_ids dominating the bottom of the rule table.** + Every row labelled `UNMATCHED` is "the LSP fired this check, no rule + modulematched, so we tagged the diagnostic with `.unmatched` + and emitted it raw". Each one is a candidate for a new rule — the + bigger `Emitted` column the more impact a rule would have. The CLAUDE.md + prompt for adding a rule lives in §4 below. +- **Knowledge gaps section says 100% coverage on every check.** + "Coverage" here means "does a rule module exist for this check name", + not "does a rule fire on every diagnostic". The two are different — + a check can have a rule module that only handles 1 of 10 sub-cases, + leaving 9 as `.unmatched`. Use the rule-performance table for the + finer-grained view. + +The rest of this document explains why each of those bullets is true. + +--- + +## 1. The big picture in three boxes + +``` +┌──────────┐ ┌─────────────────────────────────┐ ┌────────┐ +│ Agent │ → │ validate_code (one tool call) │ → │ Agent │ +└──────────┘ │ │ └────────┘ + │ 1. parse → AST │ + │ 2. lint → raw diagnostics │ + │ 3. enrich → hint, fix, conf. │ + │ 4. pipeline → suppress/verify │ + │ 5. CAC gate (optional) │ + │ 6. shape response, log emit │ + └─────────────────────────────────┘ + │ + ▼ + ┌────────────────────────────────┐ + │ Analytics (closed loop) │ + │ validator_emit → SQLite │ + │ next call → window classifier │ + │ → outcomes → case base │ + │ → engine state (next emit) │ + └────────────────────────────────┘ +``` + +Three things are happening at once: + +1. **Synchronous** — the agent's `validate_code` call walks a fixed + pipeline and gets back errors, warnings, fixes, and a + `must_fix_before_write` boolean. +2. **Persistent** — every emit is appended to + `.pos-supervisor/sessions//events.ndjson`, then ingested into + `analytics.db` (SQLite) for analysis. +3. **Reflective** — the next time the same diagnostic fires, the + engine reads the analytics and adjusts: lower confidence, suppress, + downgrade severity, or auto-disable the rule. + +Boxes (2) and (3) are the "neuro" half of the +neuro-symbolic split that the codebase calls the **adaptive engine** +(see §6). + +--- + +## 2. Vocabulary you must internalise first + +Mixing these up is the main reason the dashboard feels confusing. + +| Term | What it actually is | Lives in | +| --------------- | ----------------------------------------------------------------------------------- | ----------------------------------------- | +| **check** | The name of an issue category emitted by the LSP / pos-cli check / our structural checks. Examples: `MissingPartial`, `LiquidHTMLSyntaxError`, `pos-supervisor:HtmlInPage`. The LSP picks it. | LSP / structural-warnings.js | +| **diagnostic** | One concrete instance of a check at a (file, line, column) with a message. | the LSP / our pipeline | +| **rule** | A piece of code (`src/core/rules/.js`) that turns a raw diagnostic into a richer one with a hint, suggested fix, and confidence number. Each check has 0..N rules; the engine picks one (first-match-wins, by priority). | rules/ | +| **rule_id** | The id the rule stamps onto the diagnostic — `MissingPartial.invalid_lib_prefix`. The dashboard groups by this. | set by `apply()` of the rule | +| **`.unmatched`** | A synthetic rule_id we stamp when no rule matched, so analytics bucket every emit. Tells you: "this check fired and our rule library had nothing to say". | populated in `populateDefaultConfidence()` in diagnostic-pipeline.js | +| **hint** | A markdown blob that explains the issue to the agent. Either inline (from the rule's `apply()`) or rendered from a template under `src/data/hints/.md`. | hints/ + rules | +| **fix** | A structured proposal — `text_edit`, `insert`, `create_file`, or `guidance`. Generated by `fix-generator.js` for a fixed set of checks; rules can also return their own. | fix-generator.js + rules | +| **outcome** | What happened between two consecutive `validate_code` calls on the same file: `resolved`, `regressed`, `unchanged`, `moved`. Computed by the **window classifier**. | window-classifier.js | +| **fix_applied** | One of `verbatim`, `partial`, `ignored`, computed by comparing the file before/after the edit against any proposed fixes. | window-classifier.js → analytics-store.js | +| **window** | A pair of consecutive validate_code calls on the same (session, file). The unit of measurement. | window-classifier.js | +| **window_id** | Primary key of a `windows` row. | analytics-store.js | +| **fp** / **template_fp** | Stable hashes — `fp = hash(check, file, message_template)`, `template_fp = hash(check, message_template)`. `fp` lets us track the same diagnostic across calls; `template_fp` groups variants of the same template. | diagnostic-record.js | +| **collateral** | When a fix is applied and the diagnostic resolves, but a NEW diagnostic appears at the same time — the agent broke something else. `collateral_added` = max(0, regressed - resolved) within the same window. | window-classifier.js | +| **active rule** | A rule that is currently being run on incoming diagnostics. Same set as `_registry minus _disabledRules`. | engine.js | +| **adoption rate** | Of the windows where a fix was proposed for a diagnostic, how many ended with `fix_applied = 'verbatim'`. Per rule_id. | case-base.js | +| **resolution rate** | Of the windows where a diagnostic with this rule_id was emitted, how many ended with `outcome = 'resolved'`. | case-base.js / analytics-queries.js | +| **regression rate** | Of the same population, how many ended with `outcome = 'regressed'` (the diagnostic came back at a different fp). | case-base.js / analytics-queries.js | +| **effectiveness** | `resolution_rate - regression_rate`. The headline number on the dashboard. | analytics-labels.js | + +Internalise that list and the rest reads itself. + +--- + +## 3. The synchronous request lifecycle + +This is what happens during one `validate_code` call. The actual code +lives in `src/tools/validate-code.js` and `src/core/diagnostic-pipeline.js`. + +### 3.1 Step 1 — parse + +``` +content (string) + └─→ parseLiquidFile(content) # @platformos/liquid-html-parser + └─→ extractAllFromAST(ast) # slug, layout, method, renders, + # graphql, filters, tags, doc_params, … +``` + +We parse with the platformOS Liquid parser in **tolerant** mode, walk +the AST once with `liquid-parser.js:walk`, and produce a +`structural` object that downstream steps read. This is the "ground +truth" view of what the file actually does — slugs, methods, doc params, +referenced partials, translations. + +If the parse fails entirely, we still continue: the linter step often +catches the underlying syntax error and we want the agent to see *that* +error, not a cascade of "could not parse" infos. + +### 3.2 Step 2 — lint (raw diagnostics) + +Two upstream sources, picked at runtime: + +- **LSP path** (default when `pos-cli lsp` is up). We forward + `textDocument/didOpen` with the in-memory content, await + `publishDiagnostics`, normalise into our internal diagnostic shape + (`{ check, severity, message, line, column, endLine, endColumn, + _filePath }`). +- **check-runner fallback** (`pos-cli check run` subprocess). Used + when the LSP isn't running or crashed. Same diagnostic shape after + `parseCheckResult`. + +This step is the *only* place where check names enter the system. The +universe of check names is defined upstream by `pos-cli`, plus our own +`pos-supervisor:*` namespace from `structural-warnings.js`. + +### 3.3 Step 3 — enrich (rule engine + per-check fallbacks) + +Run inside `error-enricher.js:enrichAll`. For every diagnostic: + +1. **LSP hover** at the diagnostic position is attached as + `hover_docs`. Cached per (line, column) so duplicates are cheap. +2. **Rule engine** (`runRules(diag, facts)`): + - The rule registry is keyed by check name. For + `MissingPartial`, it loads everything in + `src/core/rules/MissingPartial.js` (priority 5, 10, 20, 30, 40) + plus any **promoted rules** from + `.pos-supervisor/promoted-rules.json` (see §6.4) and any + `_disabledRules` are skipped. + - First rule whose `when(diag, facts)` returns truthy wins. Its + `apply(diag, facts)` returns `{ rule_id, hint_md, fixes, + confidence, see_also?, case_base_signal? }`. + - The result is folded into the diagnostic. If the check has rules + and a rule matched, we *skip* the per-check regex enrichment that + follows; the rule is authoritative. +3. **Per-check regex enrichment fallback** (the older code path + that still handles ~half the checks). For checks like + `UnknownFilter`, `UndefinedObject`, `MetadataParamsCheck` etc., we + parse the raw LSP message with regexes, look up the symbol in our + indexes (`filtersIndex`, `objectsIndex`, `tagsIndex`, + `schemaIndex`), and produce a hint by rendering the appropriate + template under `src/data/hints/.md` with the extracted + variables. Shopify contamination detection happens here too — + `isShopifyObject` / `isShopifyFilter` against + `src/data/knowledge.json` (and the dedicated + `shopify-objects.json` / `shopify-contamination.json`). +4. **Pinned see-also** — `attachSeeAlso` looks up the diagnostic in + `src/data/checks/.yml` for a curated "see also" link to + another tool (e.g. `domain_guide(commands, api)`). + +After this step every diagnostic has a `hint`, possibly a +`suggestion`, a `rule_id` (or no rule_id yet — that gets stamped later +in step 5), a `confidence` (or null), and possibly `fixes`. + +### 3.4 Step 4 — diagnostic post-processing pipeline + +`runDiagnosticPipeline()` in `src/core/diagnostic-pipeline.js`. This is +where we suppress, downgrade, or annotate diagnostics that are +known-false-positive for platformOS-specific reasons. Each step is a +pure function over the result; the order is documented in the +ORDERING CONTRACT comment at the top of the file. + +The current pipeline (post our most recent fix) is: + +``` +0. userSuppressions # .pos-supervisor-ignore.yml +0a. suppressLspKnownFalsePositives # NEW: assign x = a == b regression +1. suppressDocParams # @param X declared → no UndefinedObject(X) +2. suppressUnusedDocParams # X used as named arg → not "unused" +3. elevateShopify # Shopify-* warnings → errors +4. deduplicateArgChecks # MissingRender* covers MetadataParams* +5. suppressUndocumentedTargetParams # MetadataParamsCheck on undocumented partial +6. suppressRequiredParamsWithDefault # | default:'' in target → "required" is wrong +7. suppressModuleHelpers # DeprecatedTag on module/* includes +8. suppressOrphanedPartial # commands/queries are invoked dynamically +9. suppressByPending (files) # MissingPartial for in-plan files +10. suppressByPending (pages) # MissingPage for in-plan pages +11. suppressByPending (translations) # TranslationKeyExists for in-plan keys +12. verifyMissingAssets # disk scan vs LSP cache +13. verifyTranslationKeysOnDisk # disk scan vs LSP cache +14. verifyPageRoutesOnDisk # NEW: also folds in in-memory overlay +15. verifyOrphanedPartialOnDisk # disk scan finds callers +16. verifyMissingPartialsOnDisk # disk scan vs LSP cache +17. populateDefaultConfidence # stamp .unmatched + default conf +``` + +Each step emits at most one `pos-supervisor:*Suppressed` info +diagnostic so the agent sees a single audit line per kind of +suppression instead of being silently denied. + +The ORDERING CONTRACT exists because some steps depend on others +having run already (e.g. the disk-verification steps run *after* the +in-plan suppression so an in-plan file isn't double-counted). + +### 3.5 Step 5 — fix generation + +For full mode, `fix-generator.js:generateFixes` walks the surviving +diagnostics and tries to attach a concrete `proposed_fixes` array. +Four fix kinds: + +- **`text_edit`** — exact range replacement. Used for variable + renames, filter renames, slug normalisation, etc. +- **`insert`** — insert text at a position. Used for `{% doc %}` + blocks, frontmatter additions. +- **`create_file`** — create a missing file. Used for + `MissingPartial` / `MissingAsset` when the path is unambiguous. +- **`guidance`** — description only, no machine-applicable edit. Used + when the right answer requires reasoning the linter can't do. + +A "scorecard" is also computed in full mode — a small array of +`{ category, status, reason }` rows showing how the file scores against +architectural concerns (e.g. doc-block coverage, layout +correctness). It's displayed in the agent's response and also stored +for later analysis. + +### 3.6 Step 6 — CAC predictor (optional gate) + +`cac-predictor.js:applyCac` runs over the surviving diagnostics if the +operator has enabled it (state lives in +`.pos-supervisor/cac-config.json`). For each surviving diagnostic it +predicts the probability the agent will adopt the fix, then either +allows, downgrades severity, or suppresses. Detail in §6.5. + +If CAC is in `shadow` mode, decisions are *recorded* but the result is +not mutated — used to pre-flight a threshold change before flipping to +`active`. + +### 3.7 Step 7 — shape the response, log the emit + +The final response includes: + +- `errors`, `warnings`, `infos` — the surviving diagnostics with all + the enrichment fields populated. +- `proposed_fixes` — the fix-generator output. +- `clusters` — diagnostics grouped by root-cause heuristic. +- `scorecard` — the architectural scorecard. +- `tips`, `domain_guide` — for full mode only. +- `structural` — what we extracted from the AST. +- `_pipelineTrace` — what each pipeline step removed (for the + dashboard's "Pipeline inspector" tab). +- `status` — `'ok' | 'warning' | 'error'`. +- `must_fix_before_write` — boolean. The single most important field + for the agent. If true, the agent is forbidden from writing the + file. Set whenever there's at least one error OR a "blocking + warning" survives (`OrphanedPartial`, `pos-supervisor:RemovedRender`, + etc. — the list is at the top of `validate-code.js`). +- `next_step` — a deterministic prose paragraph telling the agent what + to do next. + +Finally we emit per-diagnostic `validator_emit` events to the session +event log and a single `tool_call` event for the whole call. Both go to +`.pos-supervisor/sessions//events.ndjson`. + +--- + +## 4. The data files and their roles + +`src/data/` is a small read-mostly knowledge base that backs both the +synchronous validation path and the `lookup` / `domain_guide` / +`module_info` tools. Each file has a tightly defined role; mixing them +up is the main reason hints sometimes feel out of place. + +### 4.1 `src/core/rules/.js` — the rules + +What the registry calls a "rule". One JS file per check, each +exporting `rules: Rule[]` that gets loaded via +`src/core/rules/index.js:loadAllRules`. + +**A rule object:** + +```js +{ + id: 'MissingPartial.invalid_lib_prefix', + check: 'MissingPartial', + priority: 5, // lower = matched first + when: (diag, facts) => boolean, // gate predicate + apply: (diag, facts) => ({ + rule_id: 'MissingPartial.invalid_lib_prefix', + hint_md: '...markdown...', + fixes: [{ type: 'text_edit', ... }], + confidence: 0.95, // 0..1 + see_also: { tool, args, reason }, // optional + }), +} +``` + +The priority order is the load-bearing detail. The first rule whose +`when` returns truthy wins. So `MissingPartial.invalid_lib_prefix` +(priority 5) runs *before* `MissingPartial.module_path` (priority 10) +and `MissingPartial.suggest_nearest` (priority 30) — by the time +"suggest a nearest match" runs we know the path doesn't have the +known-bad `lib/` prefix. + +### 4.2 `src/data/hints/.md` — hint templates + +A markdown file rendered by `hint-loader.js:getHint`. Supports +`{{var}}` substitution. Used by the **per-check regex enrichment +fallback** path — i.e. the older code path that runs when no rule is +registered, or as a default when a rule doesn't include `hint_md`. + +Two categories of hints exist: + +- **Generic** — `MissingPartial.md`. Used as the default for the check. +- **Specialised** — `MissingPartial-invalid_lib_prefix.md`, + `MissingPartial-module.md`. Picked by the regex enrichment path + based on params extracted from the message. + +A hint can also live inline in a rule's `apply()` (the `hint_md` +field). When both exist, the rule's `hint_md` wins. As we migrate more +checks into the rule engine, the hints/ folder becomes the fallback, +not the primary surface. + +### 4.3 `src/data/checks/.yml` — check metadata + +A small YAML descriptor per check: + +```yaml +name: MissingPartial +summary: Referenced partial/command/query file does not exist +hint: + default: 'Create the missing file. Partials: …' +``` + +Used by: + +- `domain_guide` and `lookup` tools — they show the `summary` and + `hint.default` to give agents quick orientation. +- The rule-engine fallback when a rule doesn't supply `hint_md`. +- The dashboard's check inventory tab. + +This is the "TL;DR" surface for each check. The hints/ template is +where the long-form fix steps live. + +### 4.4 `src/data/knowledge.json` — pinned domain facts + +Most general-purpose data the validator needs. Top keys: + +- `version` +- `checks` — pinned per-check "summary" + "hint" objects + Shopify + contamination lists. This is consumed by `knowledge-loader.js`. +- `language_features` — same content as `language-features.yml`, + inlined for fast lookup. (See §4.7.) +- `domains` — per-domain rules and triggered gotchas (see `domain-gotchas.yml`). +- `content_triggers` — pattern → guidance (see `content-triggers.yml`). +- `modules_missing_docs` — list of module helpers known to ship without + `{% doc %}` blocks; the suppressUndocumentedTargetParams pipeline + step trusts this list. + +Edits here propagate everywhere: a new entry in `checks.UnknownFilter.shopify_filters` immediately changes how `error-enricher.js` classifies an `UnknownFilter` for `money_with_currency`. + +### 4.5 `src/data/content-triggers.yml` — pattern advisories + +A list of `{ id, pattern, message, severity, domains }` rules. Patterns +are regexes; when the file's *content* matches, the validator emits a +"tip" (advisory info diagnostic) in the response. Used for things that +don't fit the LSP's check vocabulary: + +```yaml +- id: raw_filter_xss + pattern: \|\s*raw\s*[%}] + message: 'XSS risk: | raw disables HTML escaping…' + severity: security + domains: [pages, partials, layouts] +``` + +The triggering happens in `getContentTriggers()` (called from +`validate-code.js`). These are *not* errors and do *not* contribute to +`must_fix_before_write` — they're shown under `tips:` in the response. + +### 4.6 `src/data/domain-gotchas.yml` — domain-aware reminders + +Domain-specific advisories keyed by the file's domain (which we infer +from path via `domain-detector.js`): + +```yaml +pages: + rule: 'Pages are controllers — logic only, no inline HTML…' + gotchas: + - id: pages_context_prefix + trigger: has_check:UndefinedObject + message: 'Use context.params, context.session, …' + severity: required +``` + +`trigger` decides when the gotcha fires. Three forms: + +- `always` — every validation in this domain. +- `has_check:` — only when the diagnostic list contains that + check. +- `uses_tag:` — only when the file uses that tag (e.g. `try`). + +`getTriggeredGotchas` returns the matching ones; they end up in +`domain_guide` (in full mode) and in the `domain_guide` tool's output. + +### 4.7 `src/data/language-features.yml` — Liquid feature reference + +Authoritative reference for platformOS-specific Liquid extensions: +`try_catch`, `theme_render_rc`, `liquid_doc`, hash literals, array +literals, etc. Used by `lookup` and the agent-facing domain guide. The +contents are also mirrored under `knowledge.json:language_features` so +runtime lookups are JSON-backed. + +If you add a new Liquid feature, write the entry here, regenerate +`knowledge.json` from this file, and the rest of the system picks it +up. + +### 4.8 `src/data/modules-missing-docs.json` — known undocumented helpers + +A flat list of paths under `modules/*` that the validator should treat +as "undocumented partial" without disk verification: + +```json +{ + "modules": [ + "modules/core/commands/execute", + "modules/admin-ui/views/partials/header", + ... + ] +} +``` + +The pipeline step `suppressUndocumentedTargetParams` reads this and +suppresses `MetadataParamsCheck` for any function/render call into +those paths (since the LSP would otherwise flag every required-param +case based on inferred-from-usage params, all of them false +positives). + +This file is the safety hatch for "the upstream module doesn't ship a +`{% doc %}` and we can't change the upstream module". + +### 4.9 `src/data/domains/.md` and `references/` + +Long-form documentation served by the `domain_guide` and `lookup` tools. +Not consumed by the validator's emit path — these are agent reading +material. `domains/commands.md`, `domains/pages.md`, etc. are the +canonical source for "how do you write a command, the platformOS way". + +### 4.10 `src/data/shopify-objects.json`, `shopify-contamination.json` + +Pinned lists of Shopify-only identifiers (objects, filters, tags) that +should never appear in platformOS code. Consumed by `knowledge-loader.js` +and `error-enricher.js` to elevate `UndefinedObject('product')` from +"variable not found" to "Shopify contamination — platformOS doesn't +have this object". The "elevateShopify" pipeline step turns those +warnings into errors. + +### 4.11 `src/data/resources/` + +Read once at server startup and exposed as MCP resources. Currently: +`platformos-synthesis.md`, the agent's session-startup primer. + +### Summary table + +| File / dir | Read by | Write surface for what | +| --------------------------------------- | ------------------------------------ | ---------------------------------------- | +| `src/core/rules/.js` | `rules/engine.js` | A rule that turns one check into a rich diagnostic + fix. | +| `src/data/hints/.md` | `hint-loader.js` → enricher fallback | Long-form fix steps for the agent. | +| `src/data/checks/.yml` | `domain_guide`, `lookup`, dashboard | Short summary + default hint per check. | +| `src/data/knowledge.json` | `knowledge-loader.js` | All the pinned check + domain + Shopify metadata in one place. | +| `src/data/content-triggers.yml` | `getContentTriggers()` | "When the file contains this pattern, also tell the agent X" advisories. | +| `src/data/domain-gotchas.yml` | `getTriggeredGotchas()` | Per-domain reminders, optionally gated by check or tag. | +| `src/data/language-features.yml` | `lookup`, `domain_guide` | Reference docs for platformOS Liquid extensions. | +| `src/data/modules-missing-docs.json` | suppressUndocumentedTargetParams | "Trust me, this module helper has no doc — don't flag callers". | +| `src/data/domains/.md` | `domain_guide` | Long-form domain documentation. | +| `src/data/references//` | `lookup` | Curated reference docs. | +| `src/data/shopify-objects.json` etc. | `knowledge-loader.js` | Shopify contamination detection lists. | + +--- + +## 5. What the dashboard words mean + +The dashboard (`/dashboard.html`, served from `src/dashboard.js` over +HTTP from `src/http-server.js`) summarises the analytics SQLite +database, so its terms are the analytics vocabulary plus a few labels. + +### 5.1 Outcomes (per diagnostic, per window) + +`window-classifier.js` takes two consecutive `validate_code` calls on +the same file and labels each diagnostic from the *first* call with +one of four outcomes: + +| Outcome | Meaning | +| ------------ | ---------------------------------------------------------------------- | +| `resolved` | The diagnostic's `fp` was present in the start call, absent in the end call. The agent fixed it. | +| `regressed` | A `fp` *not* in the start call appears in the end call. New diagnostic introduced. | +| `unchanged` | Same `fp` present in both start and end. The agent didn't fix it. | +| `moved` | The `template_fp` is present in both, but the `fp` changed. Same root cause, different line — usually because the agent edited surrounding code and the diagnostic shifted. | + +A fifth, `write_unverified`, exists for windows where we never saw a +follow-up call (the agent gave up or moved on). + +### 5.2 fix_applied (per outcome) + +For non-regressed outcomes, we compare the file's content range against +any `proposed_fixes` we emitted: + +| Value | Meaning | +| ---------- | ------------------------------------------------------------------------ | +| `verbatim` | The agent applied the proposed fix exactly. This is the strongest "they listened to us" signal. | +| `partial` | The fix range was modified, but not exactly as proposed. | +| `ignored` | The content in the fix range is unchanged (yet the diagnostic resolved or regressed for some other reason). | +| `null` | We didn't propose a fix for that diagnostic. | + +### 5.3 collateral + +Inside a single window: how many *new* diagnostics did the fix +introduce? `max(0, regressed - resolved)`. Used to penalise rules whose +"fix" creates more bugs than it solves. + +A rule with high effectiveness (`resolution - regression`) but high +collateral is doing more harm than it looks: each emit it resolves +also births a fresh diagnostic, just somewhere else. + +### 5.4 adoption rate + +For a rule_id over many windows: `adopted / total_outcomes` where +`adopted = COUNT(outcomes WHERE fix_applied = 'verbatim')`. "When this +rule fired and we proposed a fix, how often did the agent take it +verbatim". + +A low adoption rate doesn't directly mean the rule is wrong — sometimes +agents prefer their own phrasing — but it strongly correlates with +"the fix doesn't actually do what it claims". + +### 5.5 resolution / regression / effectiveness + +| Metric | Definition | +| ---------------- | ----------------------------------------------------------------------- | +| Resolution rate | `resolved / total_outcomes` per rule_id. "How often the diagnostic ends up fixed in the next call". | +| Regression rate | `regressed / total_outcomes` per rule_id. "How often the same rule reappears as a NEW diagnostic in the next call". (Note: a regression is on the rule_id, not necessarily the same `fp`.) | +| Effectiveness | `resolution_rate - regression_rate`. Goes from `-1` (every emit causes a regression) to `+1` (every emit ends in resolution). The headline rule-quality number. | + +### 5.6 Labels + +`src/core/analytics-labels.js` gates labels by sample size +(`LABEL_MIN_OUTCOMES = 5`) so a rule that fired once with a single +regression doesn't headline as `HARMFUL -100%`. + +**Per-check labels (scorecard):** + +| Label | Meaning | +| ------------------ | -------------------------------------------------------------------- | +| `INSUFFICIENT_DATA`| `total_outcomes < 5`. We don't know yet. | +| `GOOD` | `effectiveness > 0.5`. | +| `OK` | `0.15 < effectiveness ≤ 0.5`. | +| `LOW` | `0 ≤ effectiveness ≤ 0.15`. | +| `HARMFUL` | `effectiveness < 0`. The hint or fix is making things worse. | + +**Per-rule labels (rule-performance table):** + +| Label | Meaning | +| ------------------ | -------------------------------------------------------------------- | +| `UNMATCHED` | The rule_id is `.unmatched` — i.e. we emitted the diagnostic with no matching rule. **Always wins** even at low samples; coverage gap is actionable. | +| `INSUFFICIENT_DATA`| Real rule, but `< 5` outcomes. Wait. | +| `AT RISK` | Real rule, ≥ 5 outcomes, `effectiveness < 0.15`. Look at it. | +| `OK` | Real rule, ≥ 5 outcomes, `effectiveness ≥ 0.15`. Healthy. | + +### 5.7 Active / disabled / probation / promoted / force-disabled + +The state of a rule_id in the **rule registry** at a moment in time. +Defined in `engine.js`: + +| State | Set membership | +| ------------------- | --------------------------------------------------------- | +| **active** | In `_registry` and *not* in `_disabledRules`. Will run on the next emit. | +| **disabled** | In `_disabledRules` — the case-base auto-disabled it because effectiveness is bad. Skipped on emit. | +| **force-enabled** | In `_forceEnabled` (operator override). Runs even if `_disabledRules` lists it. | +| **force-disabled** | In `_forceDisabled` (operator kill-switch). Never runs, no matter what analytics say. | +| **probation** | A *promoted* rule's first 100 emits. If it crosses some quality bar in those 100, probation is resolved and it becomes a regular rule. Otherwise it gets demoted. | +| **promoted** | Came from `.pos-supervisor/promoted-rules.json` rather than `src/core/rules/.js`. Hand-authored or operator-promoted from a case-base suggestion. | + +Note the dashboard mixes "active rule" (the engine concept above) with +"active CSS class" (the UI concept of the currently-selected tab). On +the dashboard, "active" near a rule_id means the engine concept; near +a tab means the UI concept. + +### 5.8 since / baseline + +The dashboard's "Stats since" dropdown chooses a window: + +- **Since baseline** — the operator's chosen "fresh start" timestamp, + stored in `meta.analytics_baseline_ts`. +- **All time** — engine-state callers always read all time, regardless + of the operator's baseline. +- **Last 24h / 7d / Custom** — the obvious thing. + +Engine state (case-base auto-disable, probation resolution, CAC +predictor) deliberately does NOT respect the operator's baseline — a +narrow window can produce statistically meaningless decisions. +Reporting respects it. See the `resolveSince` contract in +`case-base.js`. + +--- + +## 6. The adaptive engine + +The adaptive engine is the closed loop: emits land in the analytics +database, the case base reads them back, and the engine adjusts its +behaviour for the next emit. + +### 6.1 Engine modes + +`src/core/engine-mode.js` defines two states stored in +`.pos-supervisor/engine-mode.json`: + +- **static** — every rule fires at its raw confidence, no case-base + scoring, no auto-disable, no promoted rules. Behaves like a classic + static linter. +- **adaptive** — case-base scoring ON, auto-disable ON, promoted + rules loaded ON. + +Analytics collection happens in *both* modes — only consumption +changes. You can run static for a week, accumulate data, switch to +adaptive when the case base is dense enough to be useful. + +### 6.2 Per-emit scoring + +When a rule's `apply()` returns a result, `engine.js:applyCaseBaseScoring` +runs (only in adaptive mode): + +``` +scoreRule(store, rule_id, template_fp) + → null if < MIN_CASES (3) emits for this (rule, template) + → null if no outcomes recorded + → { adjustment: number, reason: string } +``` + +The adjustment is bounded in `[-0.3, +0.3]` and shifts the rule's +emitted `confidence`. A rule with a 90% resolution rate on a specific +template gets a +0.2 boost; a rule with a 30% resolution rate gets a +−0.2 penalty. The `case_base_signal` field on the outgoing diagnostic +records the adjustment so the dashboard can show it. + +### 6.3 Auto-disable + +`case-base.js:ruleScores` runs periodically (`server.js:syncDisabledRules`). +Any rule with `effectiveness < 0.15` *and* `total_outcomes ≥ 10` is +added to `_disabledRules`. The threshold is intentionally +conservative: 10 outcomes is enough that the Beta posterior has +collapsed from "wide" to "informative", but not so high that bad rules +linger. + +Auto-disable is *override-able* by the operator: the dashboard can +mark a rule `force_enable` (it runs even though analytics disabled it) +or `force_disable` (it never runs, even if a rule module re-registers +it). Both override sets are persisted in +`.pos-supervisor/rule-overrides.json`. + +### 6.4 Promoted rules + +`src/core/rules/promoted-rules.js` loads +`.pos-supervisor/promoted-rules.json` — declarative rules entered +through the dashboard's "Suggestion → Promote" flow. The flow is: + +1. Case base finds a `template_fp` with consistent agent behaviour but + no matching rule (`synthesizeGuardPredicate`). +2. The dashboard suggests "consider adding a rule for this pattern". +3. The operator reviews and clicks Promote, optionally tweaking the + guard / hint. +4. The promoted rule lands in `promoted-rules.json` and becomes part + of the registry on the next reload — running in **probation** for + its first ~100 emits, then either auto-resolving (effectiveness + ≥ 0.5 sustained) or auto-demoting. + +The point of the probation stage is that a hand-authored rule is a +guess based on a case-base pattern that *might* generalise. We measure +it before trusting it. + +### 6.5 CAC predictor + +CAC = "case-based action classifier". It's a *fourth* gating axis on +top of severity / static confidence / adaptive-mode scoring. + +For each surviving diagnostic post-pipeline, `applyCac`: + +1. Computes an **empirical-Bayes adoption probability** using the + hierarchical scorer in `scoreFixHelpfulness`: + - try `(rule_id, file_domain)` — most specific + - fall back to `rule_id` + - fall back to `severity` + - fall back to the prior (Beta(2,2) → 0.5) +2. Decides: `allow`, `downgrade` (severity by one step), or + `suppress` — based on `config.threshold` and `config.action`. +3. Mutates the result in `active` mode; in `shadow` mode just records + the decision. + +The classifier is *always* safe — it can only suppress or downgrade, +never produce a new diagnostic, never alter a fix proposal. If the +predictor crashes, the result passes through unchanged. + +CAC is opt-in. Default is disabled. Operators turn it on after enough +analytics accumulate to make the Bayes scorer informative. + +### 6.6 The full feedback loop, illustrated + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ t = 0 agent calls validate_code │ +│ rule R fires, confidence c=0.7, fix F proposed │ +│ validator_emit logged │ +└─────────────────────────────────────────────────────────────────┘ + │ +┌──────────────── t = 1 (ms) ▼ ─────────────────────────────────┐ +│ agent applies fix F, calls validate_code on the file again │ +│ pos-supervisor runs window classifier: │ +│ start_diags ∋ {fp_X} end_diags ∌ {fp_X} │ +│ → outcome[fp_X] = 'resolved', fix_applied = 'verbatim' │ +│ one row in `outcomes` table │ +└─────────────────────────────────────────────────────────────────┘ + │ +┌──────────────── t = 2 (next call by anyone) ▼ ─────────────────┐ +│ rule R fires again on a different file, same template_fp │ +│ case base looks up scoreRule(R, template_fp): │ +│ stats: 3 emits, 3 resolved, 0 regressed → adjustment +0.2 │ +│ diagnostic confidence boosted to 0.9 │ +└─────────────────────────────────────────────────────────────────┘ + │ +┌──────────────── t = N (hours later, > 10 emits) ▼ ─────────────┐ +│ server.js:syncDisabledRules runs case-base.ruleScores │ +│ rule R: effectiveness 0.85, n=12 → not disabled │ +│ rule R': effectiveness -0.4, n=15 → added to _disabledRules │ +│ → R' won't fire on the next emit until operator overrides │ +└─────────────────────────────────────────────────────────────────┘ +``` + +Three timescales: per-emit scoring (`scoreRule`), per-batch +auto-disable (`ruleScores`), and ad-hoc promotion / probation. They +are all reading the same `outcomes` table, just at different +aggregation levels. + +--- + +## 7. How an error becomes a hint, fix, and explanation — worked example + +Take the case the agent hit in DEMO: `MissingPartial` for +`'lib/queries/contact_submissions/create'`. Trace it: + +1. **LSP** fires `MissingPartial` with message + `"'lib/queries/contact_submissions/create' does not exist"`. +2. **`normalizeLspDiagnostics`** turns the LSP shape into our internal + diagnostic `{ check: 'MissingPartial', message: '…', line, column, + _filePath }`. +3. **`enrichAll`** runs: + - `extractParams('MissingPartial', message)` extracts + `{ partial: 'lib/queries/contact_submissions/create' }`. + - `templateOf` produces a fingerprintable template: + `"'' does not exist"`. + - `runRules(diag, facts)` walks rules in priority order: + - `MissingPartial.invalid_lib_prefix` (priority 5) — + `when(diag)` checks if the partial path starts with `lib/commands/` + or `lib/queries/`. It does. Wins. + - `apply(diag)` builds the hint: "Drop the invalid `lib/` + prefix… Use `queries/contact_submissions/create` instead." A + text_edit fix is generated that removes the `lib/` prefix from + the source. + - The diagnostic's `rule_id` becomes + `MissingPartial.invalid_lib_prefix`, `confidence` is set to + `0.95`, `fixes` includes the text_edit, `hint` is the markdown. +4. **Pipeline** runs: + - `userSuppressions`: not configured. Pass. + - `suppressLspKnownFalsePositives`: only matches + `LiquidHTMLSyntaxError`. Pass. + - … + - `suppressByPending(MissingPartial)`: looks up the partial name + against `buildPendingPartialNames(pendingFiles)`. The pending list + contains `app/lib/queries/contact_submissions/create.liquid`, + which expands to short-name `queries/contact_submissions/create`. + The diagnostic's name is `lib/queries/...` — does NOT match. Pass + (correctly). + - `verifyMissingPartialsOnDisk`: tries + `app/lib/lib/queries/contact_submissions/create.liquid` — does + not exist. Confirms the LSP. Pass. + - `populateDefaultConfidence`: rule already set rule_id and + confidence, so this is a no-op. +5. **Fix generator** sees the rule already provided `fixes`, so it + doesn't add anything else. +6. **CAC** (if enabled) looks up + `(MissingPartial.invalid_lib_prefix, file_domain=pages)` history. + Suppose 4 prior emits, 4 verbatim adoptions → high probability, + `allow`. +7. **Response shape**: + - `errors[0]` = the enriched diagnostic, with hint, suggestion, + rule_id, confidence, fixes, hover_docs. + - `must_fix_before_write` = true. + - `next_step` tells the agent to apply the fix and re-validate. +8. **Emit log**: a `validator_emit` event is written with `fp`, + `template_fp`, `rule_id`, `proposed_fixes` info; a `tool_call` + event wraps the whole call. +9. **Next call** on the same file — agent has dropped the `lib/` + prefix. Window classifier sees `fp_X` absent in the end set → + `outcome = 'resolved'`, `fix_applied = 'verbatim'` (assuming the + text edit was applied as proposed). Both go into `outcomes`. +10. **Aggregation**: case-base sees this rule's effectiveness inch + up. Future emits of the same template_fp get a small confidence + boost via `scoreRule`. + +That same trace applies, with different rules selected at step 3, to +every diagnostic the system emits. The skeleton is uniform; only the +rule logic and the data files behind the hints change per check. + +--- + +## 8. Where the gaps are right now + +Reading from the report and the codebase together: + +1. **Rule coverage on `PartialCallArguments`.** 49 of 80 emits are + `.unmatched`. The rule module exists (5 priorities) but it covers + `required_render` / `required_function` / `unknown_render` / + `unknown_function` / a default — not the full surface of upstream + messages. Adding a few targeted variants would shave that + `.unmatched` count substantially. +2. **`MissingPage` resolution rate.** The bulk of the 25% number is + the self-page false positive we just fixed. The next run should + show this climbing toward `MissingPartial`'s level. +3. **`OrphanedPartial` resolution rate.** 50%. The pipeline already + suppresses orphan flags on commands/queries; the surviving 50% are + real partials that the agent doesn't always know how to wire. A + concrete `OrphanedPartial.` rule with a "where could this + be rendered from?" suggestion would help. +4. **`pos-supervisor:NonGetRenderingPage`** — the `get_form_target` + variant has 1 emit / 100% regression in the report. It's a known + pattern (form action pointing at a GET-only page) and the rule + should produce a much more specific fix proposal. +5. **`UnusedAssign.generic`** — 12 emits, 83% resolved, but the + suggestion is currently generic. A "if this is intentional, prefix + with `_`" hint would close the rest. +6. **CAC adoption is at the prior** (`feature: prior, p_adopted: 0.5`) + for most rules in the DEMO data. We need ~50+ outcomes per + `(rule_id, domain)` before CAC has signal — keep collecting before + flipping to `active`. +7. **Probation tracking is implemented but the dashboard's + "Suggestion → Promote" flow is sparse** — case-base + `synthesizeGuardPredicate` exists but the UI for reviewing + suggestions could be tighter; that's where most of the new-rule + throughput should come from once data accumulates. + +The actionable work is at the rule layer, not the pipeline layer: +every `.unmatched` row in the rule-performance table is a missing rule +in `src/core/rules/.js`. Pick the rows with the highest +`Emitted` count and write rules for them. + +--- + +## 9. Putting it all together + +To restate the system in one paragraph: + +The validator's *symbolic* core is `validate_code` walking a fixed +pipeline (parse → lint → enrich → suppress/verify → fix → respond), +backed by `src/core/rules/` for per-check logic and `src/data/` for the +domain knowledge those rules read. Its *neural* side is the analytics +loop: every emit is logged, the window classifier turns consecutive +calls into outcomes, the case base aggregates outcomes into per-rule +effectiveness, and the engine reads that back to score, disable, or +override rules on the next call. The dashboard is a window into the +analytics — labels like `GOOD`, `AT RISK`, `UNMATCHED` summarise the +case base's view of whether a rule is helping or hurting. CAC is a +fourth axis that uses the same analytics to predict whether the agent +will adopt the proposed fix at all, allowing the validator to suppress +diagnostics whose fix rarely lands. + +The improvement levers, in order of ROI: + +1. **Write rules for `.unmatched` rows** with high emit counts. +2. **Tighten hints for `LOW` and `HARMFUL` checks** — those are + actively misleading agents. +3. **Watch `regression_rate` over time** — a rule with a rising + regression rate has a hidden bug in its fix proposal. +4. **Promote case-base suggestions** through the dashboard once + `synthesizeGuardPredicate` surfaces them — this is the rule- + authoring channel that scales without manual code review. +5. **Flip CAC to `active` only after** `(rule_id, domain)` history has + ≥ 50 outcomes per cohort. Until then, CAC's prediction is the + prior and adds nothing. + +If you only remember three things from this document: + +- **The pipeline is symbolic and ordered**; every step is a documented + function and the order is load-bearing. +- **`.unmatched` is the actionable signal**; every row tells you + exactly which `src/core/rules/.js` needs a new rule. +- **Effectiveness is the only number that matters**, and it's gated by + `LABEL_MIN_OUTCOMES = 5`. Anything `INSUFFICIENT_DATA` is just + noise — wait for more data. diff --git a/package-lock.json b/package-lock.json index d36b9a9..184a65a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,15 +1,15 @@ { "name": "@platformos/pos-supervisor", - "version": "0.1.0", + "version": "0.7.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@platformos/pos-supervisor", - "version": "0.1.0", + "version": "0.7.1", "dependencies": { "@modelcontextprotocol/sdk": "^1.29.0", - "@platformos/liquid-html-parser": "^0.0.11", + "@platformos/liquid-html-parser": "^0.0.17", "js-yaml": "^4.1.1", "zod": "^4.3.6" }, @@ -76,7 +76,9 @@ } }, "node_modules/@platformos/liquid-html-parser": { - "version": "0.0.11", + "version": "0.0.17", + "resolved": "https://registry.npmjs.org/@platformos/liquid-html-parser/-/liquid-html-parser-0.0.17.tgz", + "integrity": "sha512-JhoWMZahnq28kehdQBH8Up2jB4q+h0lTlFLsZDzm1YzzlAEYmYZXjn3CK1IZ8F2zEYCkVlEYKpNkERjZVPORCA==", "license": "MIT", "dependencies": { "line-column": "^1.0.2", diff --git a/package.json b/package.json index 3e212ce..023a210 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@platformos/pos-supervisor", - "version": "0.5.2", + "version": "0.7.3", "description": "platformOS domain-specific MCP server for LLM agents", "type": "module", "bin": { @@ -13,7 +13,7 @@ }, "dependencies": { "@modelcontextprotocol/sdk": "^1.29.0", - "@platformos/liquid-html-parser": "^0.0.11", + "@platformos/liquid-html-parser": "^0.0.17", "js-yaml": "^4.1.1", "zod": "^4.3.6" }, diff --git a/scripts/cleanup-live-console-rows.js b/scripts/cleanup-live-console-rows.js new file mode 100644 index 0000000..44e282e --- /dev/null +++ b/scripts/cleanup-live-console-rows.js @@ -0,0 +1,94 @@ +#!/usr/bin/env bun +/** + * One-off cleanup — remove analytics rows that originated from the dashboard + * Live Diagnostic Console before A3 introduced the `untracked` gate. + * + * Symptom those rows caused: `__pos_live_console__` files appearing in the + * `OrphanedPartial` and `pos-supervisor:MissingDocBlock` file distributions + * of the supervisor report. Every live-console validation wrote a + * `validator_emit` that the store replayed into diagnostics/outcomes. + * + * Usage: + * bun scripts/cleanup-live-console-rows.js [/path/to/project] + * + * Project path defaults to POS_SUPERVISOR_PROJECT_DIR or the current working + * directory. Runs against `.pos-supervisor/analytics.db` under that project. + * + * Safe to re-run — purely a DELETE of rows whose file column matches the + * live-console sentinel. No schema changes. + */ + +import { join, resolve } from 'node:path'; +import { existsSync } from 'node:fs'; +import { openAnalyticsStore } from '../src/core/analytics-store.js'; + +const LIVE_CONSOLE_NEEDLE = '__pos_live_console__'; + +function parseArgs() { + const projectArg = process.argv[2]; + const projectDir = resolve(projectArg ?? process.env.POS_SUPERVISOR_PROJECT_DIR ?? process.cwd()); + return { projectDir }; +} + +function main() { + const { projectDir } = parseArgs(); + const dbPath = join(projectDir, '.pos-supervisor', 'analytics.db'); + + if (!existsSync(dbPath)) { + console.error(`No analytics DB at ${dbPath}. Nothing to clean.`); + process.exit(0); + } + + const store = openAnalyticsStore(dbPath); + const db = store.db; + + const beforeCounts = { + events: db.prepare(`SELECT COUNT(*) AS n FROM events WHERE payload LIKE ?`).get(`%${LIVE_CONSOLE_NEEDLE}%`).n, + diagnostics: db.prepare(`SELECT COUNT(*) AS n FROM diagnostics WHERE file LIKE ?`).get(`%${LIVE_CONSOLE_NEEDLE}%`).n, + outcomes: db.prepare(`SELECT COUNT(*) AS n FROM outcomes WHERE file LIKE ?`).get(`%${LIVE_CONSOLE_NEEDLE}%`).n, + windows: db.prepare(`SELECT COUNT(*) AS n FROM windows WHERE file LIKE ?`).get(`%${LIVE_CONSOLE_NEEDLE}%`).n, + proposed_fixes: db.prepare( + `SELECT COUNT(*) AS n FROM proposed_fixes pf + WHERE EXISTS (SELECT 1 FROM diagnostics d WHERE d.fp = pf.fp AND d.file LIKE ?)`, + ).get(`%${LIVE_CONSOLE_NEEDLE}%`).n, + }; + + db.exec('BEGIN'); + try { + db.prepare( + `DELETE FROM proposed_fixes + WHERE fp IN (SELECT fp FROM diagnostics WHERE file LIKE ?)`, + ).run(`%${LIVE_CONSOLE_NEEDLE}%`); + + db.prepare( + `DELETE FROM outcomes WHERE file LIKE ?`, + ).run(`%${LIVE_CONSOLE_NEEDLE}%`); + + db.prepare( + `DELETE FROM windows WHERE file LIKE ?`, + ).run(`%${LIVE_CONSOLE_NEEDLE}%`); + + db.prepare( + `DELETE FROM diagnostics WHERE file LIKE ?`, + ).run(`%${LIVE_CONSOLE_NEEDLE}%`); + + db.prepare( + `DELETE FROM events WHERE payload LIKE ?`, + ).run(`%${LIVE_CONSOLE_NEEDLE}%`); + + db.exec('COMMIT'); + } catch (e) { + db.exec('ROLLBACK'); + console.error('Cleanup failed; rolled back.'); + throw e; + } + + console.log(`Removed live-console rows from ${dbPath}:`); + for (const [table, count] of Object.entries(beforeCounts)) { + console.log(` ${table.padEnd(16)} ${count}`); + } + + store.close(); +} + +main(); diff --git a/scripts/rebuild-analytics.js b/scripts/rebuild-analytics.js new file mode 100644 index 0000000..84bfe36 --- /dev/null +++ b/scripts/rebuild-analytics.js @@ -0,0 +1,57 @@ +#!/usr/bin/env node +/** + * Rebuild the analytics DB from session event logs. + * + * Usage: + * node scripts/rebuild-analytics.js /path/to/project + * + * The project must have a .pos-supervisor/ directory with sessions/ and analytics.db. + * The server must NOT be running when this script executes (WAL mode allows reads + * but schema migrations can conflict with a live server). + */ + +import { join } from 'node:path'; +import { existsSync } from 'node:fs'; +import { openAnalyticsStore } from '../src/core/analytics-store.js'; +import { openBlobStore } from '../src/core/blob-store.js'; + +const projectDir = process.argv[2]; +if (!projectDir) { + console.error('Usage: node scripts/rebuild-analytics.js /path/to/project'); + process.exit(1); +} + +const supervisorDir = join(projectDir, '.pos-supervisor'); +const dbPath = join(supervisorDir, 'analytics.db'); +const sessionsDir = join(supervisorDir, 'sessions'); +const blobsDir = join(supervisorDir, 'blobs'); + +if (!existsSync(supervisorDir)) { + console.error(`No .pos-supervisor directory found at: ${supervisorDir}`); + process.exit(1); +} +if (!existsSync(sessionsDir)) { + console.error(`No sessions directory found at: ${sessionsDir}`); + process.exit(1); +} + +console.log(`DB: ${dbPath}`); +console.log(`Sessions: ${sessionsDir}`); +console.log(`Blobs: ${blobsDir}`); +console.log('Rebuilding...'); + +// Blob store is required for fix-adoption classification (reads start/end file +// snapshots and proposed-fix texts). Without it, every outcome row lands with +// fix_applied = null. Fine if the blobs dir doesn't exist yet — classification +// just degrades to null for that session. +let blobStore = null; +try { + blobStore = openBlobStore(blobsDir); +} catch (e) { + console.warn(`Blob store unavailable (${e.message}); fix adoption will not be classified.`); +} + +const store = openAnalyticsStore(dbPath, { blobStore }); +const { sessions, events } = store.rebuild(sessionsDir); + +console.log(`Done. Replayed ${events} events across ${sessions} sessions.`); diff --git a/src/core/analytics-labels.js b/src/core/analytics-labels.js new file mode 100644 index 0000000..75311b5 --- /dev/null +++ b/src/core/analytics-labels.js @@ -0,0 +1,126 @@ +/** + * Analytics labels — single source of truth for the GOOD / OK / LOW / HARMFUL, + * AT RISK / UNMATCHED, and INSUFFICIENT_DATA presentation-layer labels. + * + * Pure functions, intentionally side-effect-free. The HTTP layer attaches + * `.label` to each scorecard / rule-performance row before serialising; the + * dashboard browser code and Markdown report consume that field directly so + * label logic isn't duplicated (or drifted) between server and client. + * + * INSUFFICIENT_DATA gate (`LABEL_MIN_OUTCOMES`) is the load-bearing change. + * Labels computed from a sample of one — `AT RISK -100%` on a single + * regression — are statistically meaningless and previously caused operators + * to chase ghosts of already-fixed rules. Below the threshold we return a + * neutral label that says "we don't know yet" instead of a confident wrong + * answer. + * + * The threshold is conservative on purpose: 5 outcomes lets a Beta(2,2) + * posterior collapse from "wide ribbon" to a meaningful interval. Engine-side + * decisions (auto-disable in case-base.ruleScores) use a stricter gate of 10 + * because promotion/demotion is more consequential than display. + */ + +export const LABEL_MIN_OUTCOMES = 5; + +/** + * Normalise a Beta-posterior object or bare number to a scalar in [0, 1]. + * Mirrors the dashboard `rateVal()` helper exactly so the server emits the + * same labels the browser would have computed inline. + */ +function asRate(r) { + if (r && typeof r === 'object' && typeof r.mean === 'number') return r.mean; + if (typeof r === 'number') return r; + return 0; +} + +/** + * Per-check scorecard label. + * + * Accepts a row from `checkScorecards()` carrying `.resolution_rate`, + * `.mislead_rate`, and either `.sample_size` (preferred) or `.total_outcomes`. + * Each rate may be a Beta posterior `{ mean, lower95, upper95 }` or a number. + * + * Returns one of: + * - INSUFFICIENT_DATA — fewer than LABEL_MIN_OUTCOMES outcomes + * - GOOD — effectiveness > 0.5 + * - OK — 0.15 < effectiveness <= 0.5 + * - LOW — 0 <= effectiveness <= 0.15 + * - HARMFUL — effectiveness < 0 + */ +export function checkLabel(card) { + if (!card || typeof card !== 'object') return 'INSUFFICIENT_DATA'; + const sampleSize = Number(card.sample_size ?? card.total_outcomes ?? 0); + if (!Number.isFinite(sampleSize) || sampleSize < LABEL_MIN_OUTCOMES) { + return 'INSUFFICIENT_DATA'; + } + const effectiveness = asRate(card.resolution_rate) - asRate(card.mislead_rate); + if (effectiveness > 0.5) return 'GOOD'; + if (effectiveness > 0.15) return 'OK'; + if (effectiveness >= 0) return 'LOW'; + return 'HARMFUL'; +} + +/** + * Per-rule_id performance label. + * + * Accepts a row from `rulePerformance()` / `ruleScores()` carrying + * `.unmatched`, `.effectiveness`, and `.total_outcomes`. + * + * Precedence: + * 1. UNMATCHED — `.unmatched === true` always wins. Coverage gap is + * actionable regardless of sample size; one emit on a + * rule-less check still tells the operator a rule needs + * writing. + * 2. INSUFFICIENT_DATA — `total_outcomes < LABEL_MIN_OUTCOMES`. We don't + * know enough to call the rule risky. + * 3. AT RISK — effectiveness < 0.15. Real signal, real concern. + * 4. OK — everything else. + * + * Note: `effectiveness` here is `resolution_rate - regression_rate`, not the + * 0..1 percentage the case-base disable-gate uses. A negative number is + * possible (rule causes more regressions than it resolves). + */ +export function ruleLabel(rule) { + if (!rule || typeof rule !== 'object') return 'INSUFFICIENT_DATA'; + if (rule.unmatched) return 'UNMATCHED'; + const totalOutcomes = Number(rule.total_outcomes ?? 0); + if (!Number.isFinite(totalOutcomes) || totalOutcomes < LABEL_MIN_OUTCOMES) { + return 'INSUFFICIENT_DATA'; + } + const effectiveness = Number(rule.effectiveness ?? 0); + if (!Number.isFinite(effectiveness)) return 'INSUFFICIENT_DATA'; + if (effectiveness < 0.15) return 'AT RISK'; + return 'OK'; +} + +/** + * Filter scorecards down to the rows that warrant a HARMFUL headline in the + * Markdown report's executive summary. Honours the same sample-size gate so + * we don't trumpet "HARMFUL" off a single regression — which is exactly the + * stale-data trap that motivated this whole module. + */ +export function harmfulSummary(scorecards) { + if (!Array.isArray(scorecards)) return []; + return scorecards.filter(c => checkLabel(c) === 'HARMFUL'); +} + +/** + * Attach a `.label` field to every row in a scorecard array. Returns a NEW + * array; rows are shallow-copied so callers can't accidentally mutate the + * underlying analytics-queries result. HTTP handlers wrap the array with this + * before sending so the dashboard receives labelled rows it can render + * without re-computing. + */ +export function withCheckLabels(scorecards) { + if (!Array.isArray(scorecards)) return []; + return scorecards.map(card => ({ ...card, label: checkLabel(card) })); +} + +/** + * Attach a `.label` field to every row in a rule-performance / rule-score + * array. See `withCheckLabels`. + */ +export function withRuleLabels(rules) { + if (!Array.isArray(rules)) return []; + return rules.map(rule => ({ ...rule, label: ruleLabel(rule) })); +} diff --git a/src/core/analytics-queries.js b/src/core/analytics-queries.js new file mode 100644 index 0000000..2403408 --- /dev/null +++ b/src/core/analytics-queries.js @@ -0,0 +1,1125 @@ +/** + * Analytics queries — scorecards, association rules, and cohort analysis. + * + * All functions take an opened analytics store and return structured results. + * Bayesian posteriors use Beta-binomial with prior Beta(2,2) — weakly + * informative, symmetric. Surface the 95% lower bound so low-sample checks + * don't appear artificially confident. + */ + +const MIN_COHORT = 10; + +function tryParseJson(str) { + if (!str) return null; + try { return JSON.parse(str); } catch { return null; } +} + +/** + * Resolve the tri-state `since` parameter to an ISO string or null. + * + * Reporting queries take an optional `since` opt. The contract: + * + * - `since === undefined` (or absent): read the store's reporting baseline + * meta (`analytics_baseline_ts`). Absent meta ⇒ null ⇒ no filter. This is + * the reporting default — operators set the baseline once and every + * dashboard widget / Markdown report widget sees the post-baseline view. + * - `since === null`: explicit bypass — never filter, regardless of meta. + * Reserved for engine-state callers that must see full history (case-base + * auto-disable, scoreRule, server-status ops snapshot). Reporting callers + * should not use this; tests use it to assert "default behaviour with no + * baseline" without depending on meta state. + * - `since === ''`: explicit override — use that timestamp. + * + * `store.getBaselineTs` is the analytics-store helper; absent (e.g. mock + * stores), the resolver degrades to "no baseline" gracefully. + */ +function resolveSince(store, since) { + if (since === null) return null; + if (typeof since === 'string' && since.length > 0) return since; + if (store && typeof store.getBaselineTs === 'function') { + try { + const baseline = store.getBaselineTs(); + return baseline ?? null; + } catch { + return null; + } + } + return null; +} + +/** + * Beta-binomial posterior: given `successes` out of `total` trials + * with prior Beta(a, b), return { mean, lower95, upper95 }. + */ +export function betaPosterior(successes, total, a = 2, b = 2) { + const postA = a + successes; + const postB = b + (total - successes); + const mean = postA / (postA + postB); + + const lower = betaQuantile(0.025, postA, postB); + const upper = betaQuantile(0.975, postA, postB); + + return { mean, lower95: lower, upper95: upper }; +} + +/** + * Approximate Beta quantile using the normal approximation to the Beta + * distribution. Exact quantile computation requires the regularized + * incomplete beta function; the normal approximation is adequate for + * dashboard display at the sample sizes we deal with (n > 10). + */ +function betaQuantile(p, a, b) { + const mean = a / (a + b); + const variance = (a * b) / ((a + b) ** 2 * (a + b + 1)); + const sd = Math.sqrt(variance); + const z = normalQuantile(p); + return Math.max(0, Math.min(1, mean + z * sd)); +} + +function normalQuantile(p) { + // Rational approximation (Abramowitz & Stegun 26.2.23) + if (p <= 0) return -Infinity; + if (p >= 1) return Infinity; + if (p === 0.5) return 0; + const sign = p < 0.5 ? -1 : 1; + const t = p < 0.5 ? p : 1 - p; + const x = Math.sqrt(-2 * Math.log(t)); + const c0 = 2.515517, c1 = 0.802853, c2 = 0.010328; + const d1 = 1.432788, d2 = 0.189269, d3 = 0.001308; + return sign * (x - (c0 + c1 * x + c2 * x * x) / (1 + d1 * x + d2 * x * x + d3 * x * x * x)); +} + +/** + * Per-check scorecard: emitted count, resolution rate, mislead rate, + * adoption rate, average collateral. + * + * @param {object} store - Opened analytics store + * @param {object} [opts] + * @param {number} [opts.minCohort=10] - Minimum sample size for inclusion + * @param {string} [opts.sessionId] - Limit to specific session + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array} + */ +export function checkScorecards(store, { minCohort = MIN_COHORT, sessionId, since } = {}) { + const sinceTs = resolveSince(store, since); + const sessionFilter = sessionId ? 'AND d.session_id = ?' : ''; + const sinceFilter = sinceTs ? 'AND d.ts >= ?' : ''; + const params = sessionId ? [sessionId] : []; + if (sinceTs) params.push(sinceTs); + + const emittedRows = store.query(` + SELECT d.check_name, COUNT(*) as emitted + FROM diagnostics d + WHERE d.suppressed = 0 ${sessionFilter} ${sinceFilter} + GROUP BY d.check_name + HAVING COUNT(*) >= ? + `, [...params, minCohort]); + + const scorecards = []; + + // The `since` filter is applied to the diagnostics subquery — we want to + // count outcomes only for diagnostics that fall in the reporting window. + // The outcomes table itself has no ts column; the diagnostic's emit ts is + // the canonical "when this happened" for reporting purposes. + const outcomeSinceClause = sinceTs ? 'AND ts >= ?' : ''; + const outcomeSinceParams = sinceTs ? [sinceTs] : []; + + for (const row of emittedRows) { + const check = row.check_name; + const emitted = row.emitted; + + const sessionDiagFilter = sessionId ? 'AND session_id = ?' : ''; + + const outcomeRows = store.query(` + SELECT o.outcome, COUNT(*) as cnt + FROM outcomes o + WHERE o.fp IN ( + SELECT fp FROM diagnostics + WHERE check_name = ? ${sessionDiagFilter} ${outcomeSinceClause} + ) + GROUP BY o.outcome + `, [check, ...(sessionId ? [sessionId] : []), ...outcomeSinceParams]); + + const outcomes = {}; + for (const r of outcomeRows) outcomes[r.outcome] = r.cnt; + + const totalOutcomes = Object.values(outcomes).reduce((s, v) => s + v, 0); + const resolved = outcomes.resolved ?? 0; + const regressed = outcomes.regressed ?? 0; + + const resolution = totalOutcomes > 0 + ? betaPosterior(resolved, totalOutcomes) + : { mean: 0, lower95: 0, upper95: 0 }; + + const mislead = totalOutcomes > 0 + ? betaPosterior(regressed, totalOutcomes) + : { mean: 0, lower95: 0, upper95: 0 }; + + const fixRows = store.query(` + SELECT o.fix_applied, COUNT(*) as cnt + FROM outcomes o + WHERE o.fp IN ( + SELECT fp FROM diagnostics + WHERE check_name = ? ${outcomeSinceClause} + ) + AND o.fix_applied IS NOT NULL + GROUP BY o.fix_applied + `, [check, ...outcomeSinceParams]); + + const fixCounts = {}; + for (const r of fixRows) fixCounts[r.fix_applied] = r.cnt; + const totalFixes = Object.values(fixCounts).reduce((s, v) => s + v, 0); + const verbatim = fixCounts.verbatim ?? 0; + + const adoption = totalFixes > 0 + ? betaPosterior(verbatim, totalFixes) + : { mean: 0, lower95: 0, upper95: 0 }; + + const collateralRow = store.queryOne(` + SELECT AVG(o.collateral_added) as avg_collateral + FROM outcomes o + WHERE o.fp IN ( + SELECT fp FROM diagnostics + WHERE check_name = ? ${outcomeSinceClause} + ) + AND o.outcome = 'regressed' + `, [check, ...outcomeSinceParams]); + + scorecards.push({ + check, + emitted, + resolution_rate: resolution, + mislead_rate: mislead, + adoption_rate: adoption, + avg_collateral: collateralRow?.avg_collateral ?? 0, + sample_size: totalOutcomes, + }); + } + + scorecards.sort((a, b) => b.mislead_rate.mean - a.mislead_rate.mean); + return scorecards; +} + +/** + * Tool-call sequence bigrams within a session. Computes lift and confidence + * for each bigram vs baseline frequency. + * + * @param {object} store - Opened analytics store + * @param {object} [opts] + * @param {string} [opts.sessionId] - Limit to specific session + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array<{bigram: [string,string], count, lift, confidence}>} + */ +export function toolSequenceBigrams(store, { sessionId, since } = {}) { + const sinceTs = resolveSince(store, since); + const clauses = []; + const params = []; + if (sessionId) { + clauses.push('session_id = ?'); + params.push(sessionId); + } + if (sinceTs) { + clauses.push('ts >= ?'); + params.push(sinceTs); + } + const where = clauses.length > 0 ? `WHERE ${clauses.join(' AND ')}` : ''; + + const events = store.query(` + SELECT kind, payload FROM events + ${where} + ORDER BY ts ASC + `, params); + + const toolCalls = []; + for (const e of events) { + if (e.kind !== 'tool_call') continue; + try { + const payload = JSON.parse(e.payload); + toolCalls.push(payload.tool); + } catch { continue; } + } + + if (toolCalls.length < 2) return []; + + const bigramCounts = new Map(); + const unigramCounts = new Map(); + + for (let i = 0; i < toolCalls.length; i++) { + unigramCounts.set(toolCalls[i], (unigramCounts.get(toolCalls[i]) ?? 0) + 1); + if (i < toolCalls.length - 1) { + const key = `${toolCalls[i]}→${toolCalls[i + 1]}`; + bigramCounts.set(key, (bigramCounts.get(key) ?? 0) + 1); + } + } + + const total = toolCalls.length; + const totalBigrams = toolCalls.length - 1; + + const results = []; + for (const [key, count] of bigramCounts) { + const [a, b] = key.split('→'); + const pA = (unigramCounts.get(a) ?? 0) / total; + const pB = (unigramCounts.get(b) ?? 0) / total; + const pAB = count / totalBigrams; + const expected = pA * pB; + const lift = expected > 0 ? pAB / expected : 0; + const confidence = (unigramCounts.get(a) ?? 0) > 0 ? count / unigramCounts.get(a) : 0; + results.push({ bigram: [a, b], count, lift, confidence }); + } + + results.sort((a, b) => b.lift - a.lift); + return results; +} + +/** + * Session-level summary: key metrics per session for cohort comparison. + * + * @param {object} store - Opened analytics store + * @param {object} [opts] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array} + */ +export function sessionSummaries(store, { since } = {}) { + const sinceTs = resolveSince(store, since); + const eventSinceWhere = sinceTs ? 'WHERE ts >= ?' : ''; + const eventSinceAnd = sinceTs ? 'AND ts >= ?' : ''; + const windowSinceAnd = sinceTs ? 'AND w.ts_start >= ?' : ''; + + // Session list: sessions that had ANY event at-or-after the baseline. The + // session's apparent first_event is naturally clamped to the window since + // MIN(ts) only sees ts >= since rows. + const sessions = store.query(` + SELECT session_id, + MIN(ts) as first_event, + MAX(ts) as last_event, + COUNT(*) as event_count + FROM events + ${eventSinceWhere} + GROUP BY session_id + ORDER BY MIN(ts) DESC + `, sinceTs ? [sinceTs] : []); + + return sessions.map(s => { + const sinceP = sinceTs ? [sinceTs] : []; + + const toolCalls = store.queryOne(` + SELECT COUNT(*) as cnt FROM events + WHERE session_id = ? AND kind = 'tool_call' ${eventSinceAnd} + `, [s.session_id, ...sinceP]); + + const vcCalls = store.queryOne(` + SELECT COUNT(*) as cnt FROM events + WHERE session_id = ? AND kind = 'tool_call' + AND payload LIKE '%"tool":"validate_code"%' ${eventSinceAnd} + `, [s.session_id, ...sinceP]); + + const diagCount = store.queryOne(` + SELECT COUNT(*) as cnt FROM diagnostics + WHERE session_id = ? ${eventSinceAnd} + `, [s.session_id, ...sinceP]); + + const outcomeRow = store.queryOne(` + SELECT COUNT(*) as total, + SUM(CASE WHEN outcome = 'resolved' THEN 1 ELSE 0 END) as resolved, + SUM(CASE WHEN outcome = 'regressed' THEN 1 ELSE 0 END) as regressed + FROM outcomes o + JOIN windows w ON o.window_id = w.id + WHERE w.session_id = ? ${windowSinceAnd} + `, [s.session_id, ...sinceP]); + + const usedIntent = store.queryOne(` + SELECT COUNT(*) as cnt FROM events + WHERE session_id = ? AND kind = 'tool_call' + AND payload LIKE '%"tool":"validate_intent"%' ${eventSinceAnd} + `, [s.session_id, ...sinceP]); + + return { + session_id: s.session_id, + first_event: s.first_event, + last_event: s.last_event, + event_count: s.event_count, + tool_calls: toolCalls?.cnt ?? 0, + validate_code_calls: vcCalls?.cnt ?? 0, + used_validate_intent: (usedIntent?.cnt ?? 0) > 0, + diagnostics_emitted: diagCount?.cnt ?? 0, + outcomes_total: outcomeRow?.total ?? 0, + outcomes_resolved: outcomeRow?.resolved ?? 0, + outcomes_regressed: outcomeRow?.regressed ?? 0, + }; + }); +} + +/** + * Identify checks with high mislead rates that warrant hint rewriting. + * + * @param {object} store + * @param {number} [threshold=0.3] - Mislead rate threshold + * @param {object} [opts] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array<{check, mislead_rate, recommendation}>} + */ +export function recommendations(store, threshold = 0.3, { since } = {}) { + const cards = checkScorecards(store, { minCohort: Math.max(MIN_COHORT, 5), since }); + const recs = []; + + for (const card of cards) { + if (card.mislead_rate.mean >= threshold) { + recs.push({ + check: card.check, + mislead_rate: card.mislead_rate.mean, + recommendation: `Check \`${card.check}\` misleads ${(card.mislead_rate.mean * 100).toFixed(0)}% of fixes — consider rewriting hint in \`src/data/hints/${card.check}.md\` or its rule in \`src/core/rules/${card.check}.js\``, + }); + } + } + + return recs; +} + +/** + * K2: Diagnostic journey — the full lifecycle of a diagnostic template + * across sessions. Shows when it first appeared, which rules fired, + * what outcomes occurred, and whether it was eventually resolved. + * + * @param {object} store + * @param {string} templateFp + * @param {object} [opts] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {{ template_fp, check, first_seen, last_seen, session_count, timeline }} + */ +export function diagnosticJourney(store, templateFp, { since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND ts >= ?' : ''; + const sinceAndD = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const meta = store.queryOne(` + SELECT check_name, + MIN(ts) as first_seen, + MAX(ts) as last_seen, + COUNT(DISTINCT session_id) as session_count + FROM diagnostics + WHERE template_fp = ? AND suppressed = 0 ${sinceAnd} + `, [templateFp, ...sinceP]); + + if (!meta || !meta.check_name) { + return { template_fp: templateFp, check: null, first_seen: null, last_seen: null, session_count: 0, timeline: [] }; + } + + const timelineRows = store.query(` + SELECT d.session_id, + d.ts, + d.hint_rule_id, + d.hint_md_hash, + d.fp, + d.content_hash, + d.file, + o.outcome, + o.fix_applied + FROM diagnostics d + LEFT JOIN outcomes o ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.template_fp = ? AND d.suppressed = 0 ${sinceAndD} + ORDER BY d.ts ASC + `, [templateFp, ...sinceP]); + + const bySession = new Map(); + for (const row of timelineRows) { + if (!bySession.has(row.session_id)) { + bySession.set(row.session_id, { + session_id: row.session_id, + ts: row.ts, + occurrences: 0, + rule_id: null, + outcomes: [], + content_hash: null, + hint_md_hash: null, + fp: null, + file: null, + }); + } + const entry = bySession.get(row.session_id); + entry.occurrences++; + if (row.hint_rule_id && row.hint_rule_id !== 'unknown') entry.rule_id = row.hint_rule_id; + if (row.outcome) entry.outcomes.push({ outcome: row.outcome, fix_applied: row.fix_applied ?? null }); + if (row.content_hash && !entry.content_hash) { + entry.content_hash = row.content_hash; + entry.fp = row.fp; + entry.file = row.file; + } + if (row.hint_md_hash && !entry.hint_md_hash) { + entry.hint_md_hash = row.hint_md_hash; + } + } + + // Fetch fix data for each session entry that has a diagnostic fingerprint. + for (const entry of bySession.values()) { + if (!entry.fp) continue; + const fixRow = store.queryOne(` + SELECT new_text_hash, range_json FROM proposed_fixes + WHERE fp = ? AND session_id = ? LIMIT 1 + `, [entry.fp, entry.session_id]); + entry.fix_hash = fixRow?.new_text_hash ?? null; + entry.fix_range = tryParseJson(fixRow?.range_json); + } + + const timeline = [...bySession.values()].map(s => { + const resolved = s.outcomes.filter(o => o.outcome === 'resolved').length; + const regressed = s.outcomes.filter(o => o.outcome === 'regressed').length; + const dominant = resolved > 0 ? 'resolved' + : regressed > 0 ? 'regressed' + : s.outcomes.length > 0 ? 'unchanged' + : null; + + return { + session_id: s.session_id, + ts: s.ts, + occurrences: s.occurrences, + rule_id: s.rule_id, + dominant_outcome: dominant, + fix_applied: s.outcomes.find(o => o.fix_applied)?.fix_applied ?? null, + content_hash: s.content_hash ?? null, + hint_md_hash: s.hint_md_hash ?? null, + fix_hash: s.fix_hash ?? null, + fix_range: s.fix_range ?? null, + file: s.file ?? null, + }; + }); + + return { + template_fp: templateFp, + check: meta.check_name, + first_seen: meta.first_seen, + last_seen: meta.last_seen, + session_count: meta.session_count, + timeline, + }; +} + +/** + * K3: Confidence calibration — compare predicted confidence to actual + * resolution rates. Buckets confidence values and computes actual + * outcomes for each bucket. + * + * @param {object} store + * @param {object} [opts] + * @param {number} [opts.buckets=10] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array<{ bucket, predicted, actual_resolution, sample_size }>} + */ +export function confidenceCalibration(store, { buckets = 10, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + // Post-A2: every surviving diagnostic gets a default confidence in the + // pipeline, so dropping the `confidence IS NOT NULL` guard widens the + // calibration sample to cover non-rule-matched diagnostics too. Rows + // predating A2 (no pipeline default) will have NULL confidence — exclude + // those explicitly so the bucketing math doesn't see NaN. + const rows = store.query(` + SELECT d.confidence, o.outcome + FROM diagnostics d + JOIN outcomes o ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.suppressed = 0 AND d.confidence IS NOT NULL ${sinceAnd} + `, sinceP); + + if (rows.length === 0) return []; + + const bucketWidth = 1.0 / buckets; + const bucketData = Array.from({ length: buckets }, (_, i) => ({ + lower: i * bucketWidth, + upper: (i + 1) * bucketWidth, + resolved: 0, + total: 0, + })); + + for (const row of rows) { + const idx = Math.min(Math.floor(row.confidence / bucketWidth), buckets - 1); + bucketData[idx].total++; + if (row.outcome === 'resolved') bucketData[idx].resolved++; + } + + return bucketData + .filter(b => b.total > 0) + .map(b => ({ + bucket: +((b.lower + b.upper) / 2).toFixed(2), + predicted: +((b.lower + b.upper) / 2).toFixed(2), + actual_resolution: +(b.resolved / b.total).toFixed(4), + sample_size: b.total, + })); +} + +/** + * K4: Fix adoption funnel — aggregate flow from diagnostic emission + * through rule matching, fix proposal, adoption, and resolution. + * + * @param {object} store + * @param {object} [opts] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {{ emitted, rule_matched, fix_proposed, fix_adopted_verbatim, + * fix_adopted_partial, fix_ignored, resolved, regressed, unchanged }} + */ +export function fixAdoptionFunnel(store, { since } = {}) { + const sinceTs = resolveSince(store, since); + // Same `ts` column on diagnostics; use plain `ts >=` outside the EXISTS. + const sinceAndPlain = sinceTs ? 'AND ts >= ?' : ''; + // Inside EXISTS subqueries we alias diagnostics as `d`. + const sinceAndD = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const emittedRow = store.queryOne(` + SELECT COUNT(*) as cnt FROM diagnostics + WHERE suppressed = 0 ${sinceAndPlain} + `, sinceP); + const emitted = emittedRow?.cnt ?? 0; + + const ruleMatchedRow = store.queryOne(` + SELECT COUNT(*) as cnt FROM diagnostics + WHERE hint_rule_id IS NOT NULL AND hint_rule_id != 'unknown' AND suppressed = 0 ${sinceAndPlain} + `, sinceP); + const rule_matched = ruleMatchedRow?.cnt ?? 0; + + const fixProposedRow = store.queryOne(` + SELECT COUNT(DISTINCT pf.fp) as cnt + FROM proposed_fixes pf + JOIN diagnostics d ON pf.fp = d.fp + WHERE d.suppressed = 0 ${sinceAndD} + `, sinceP); + const fix_proposed = fixProposedRow?.cnt ?? 0; + + // Post-A1 (dedup): outcomes has one row per (session, file, fp). A plain + // JOIN to diagnostics on fp cross-joins by emit count — use EXISTS to + // keep the count at one-per-outcome-row. Baseline filter goes inside the + // EXISTS so an outcome only counts when its diagnostic falls in the + // reporting window. + const fixAdoptionRows = store.query(` + SELECT o.fix_applied, COUNT(*) as cnt + FROM outcomes o + WHERE o.fix_applied IS NOT NULL + AND EXISTS (SELECT 1 FROM diagnostics d WHERE d.fp = o.fp AND d.suppressed = 0 ${sinceAndD}) + GROUP BY o.fix_applied + `, sinceP); + let fix_adopted_verbatim = 0, fix_adopted_partial = 0, fix_ignored = 0; + for (const row of fixAdoptionRows) { + if (row.fix_applied === 'verbatim') fix_adopted_verbatim += row.cnt; + else if (row.fix_applied === 'partial') fix_adopted_partial += row.cnt; + else fix_ignored += row.cnt; + } + + const outcomeRows = store.query(` + SELECT o.outcome, COUNT(*) as cnt + FROM outcomes o + WHERE EXISTS (SELECT 1 FROM diagnostics d WHERE d.fp = o.fp AND d.suppressed = 0 ${sinceAndD}) + GROUP BY o.outcome + `, sinceP); + let resolved = 0, regressed = 0, unchanged = 0; + for (const row of outcomeRows) { + if (row.outcome === 'resolved') resolved += row.cnt; + else if (row.outcome === 'regressed') regressed += row.cnt; + else if (row.outcome === 'unchanged') unchanged += row.cnt; + } + + return { + emitted, + rule_matched, + fix_proposed, + fix_adopted_verbatim, + fix_adopted_partial, + fix_ignored, + resolved, + regressed, + unchanged, + }; +} + +/** + * L5: Rule effectiveness broken down by file category (pages, partials, + * commands, queries, graphql). Used by the heatmap visualization. + * + * @param {object} store + * @param {object} [opts] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array<{ rule_id, check, category, outcomes, resolved, regressed, effectiveness }>} + */ +export function ruleScoresByCategory(store, { since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const rows = store.query(` + SELECT d.hint_rule_id as rule_id, + d.check_name, + d.file, + o.outcome + FROM diagnostics d + JOIN outcomes o ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.hint_rule_id IS NOT NULL AND d.hint_rule_id != 'unknown' AND d.suppressed = 0 ${sinceAnd} + `, sinceP); + + const buckets = new Map(); + for (const row of rows) { + const cat = classifyFilePath(row.file); + const key = row.rule_id + '::' + cat; + if (!buckets.has(key)) { + buckets.set(key, { rule_id: row.rule_id, check: row.check_name, category: cat, outcomes: 0, resolved: 0, regressed: 0 }); + } + const b = buckets.get(key); + b.outcomes++; + if (row.outcome === 'resolved') b.resolved++; + else if (row.outcome === 'regressed') b.regressed++; + } + + return [...buckets.values()].map(b => ({ + ...b, + effectiveness: b.outcomes > 0 + ? +((b.resolved / b.outcomes) - (b.regressed / b.outcomes)).toFixed(4) + : 0, + })); +} + +function classifyFilePath(file) { + if (!file) return 'other'; + if (file.startsWith('app/views/pages/') || file.startsWith('app/views/layouts/')) return 'pages'; + if (file.startsWith('app/views/partials/')) return 'partials'; + if (file.startsWith('app/lib/commands/') || file.includes('/mutations/')) return 'commands'; + if (file.startsWith('app/lib/queries/')) return 'queries'; + if (file.endsWith('.graphql') || file.startsWith('app/graphql/')) return 'graphql'; + if (file.startsWith('app/schema/') || file.endsWith('.yml') || file.endsWith('.yaml')) return 'schema'; + return 'other'; +} + +/** + * K5: Knowledge gaps — identify checks where rule coverage is low + * (diagnostics with no matching rule). Helps prioritize rule writing. + * + * @param {object} store + * @param {object} [opts] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array<{ check, unmatched_count, total_emitted, coverage_rate, avg_resolution_rate }>} + */ +export function knowledgeGaps(store, { since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND ts >= ?' : ''; + const sinceAndD = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const checkRows = store.query(` + SELECT check_name, + COUNT(*) as total_emitted, + SUM(CASE WHEN hint_rule_id IS NULL OR hint_rule_id = 'unknown' THEN 1 ELSE 0 END) as unmatched + FROM diagnostics + WHERE suppressed = 0 ${sinceAnd} + GROUP BY check_name + HAVING COUNT(*) >= 3 + ORDER BY total_emitted DESC + `, sinceP); + + return checkRows.map(row => { + const resRow = store.queryOne(` + SELECT COUNT(*) as total, + SUM(CASE WHEN o.outcome = 'resolved' THEN 1 ELSE 0 END) as resolved + FROM outcomes o + JOIN diagnostics d ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.check_name = ? AND d.suppressed = 0 ${sinceAndD} + `, [row.check_name, ...sinceP]); + + const totalOutcomes = resRow?.total ?? 0; + const resolvedCount = resRow?.resolved ?? 0; + + return { + check: row.check_name, + unmatched_count: row.unmatched, + total_emitted: row.total_emitted, + coverage_rate: row.total_emitted > 0 ? +((row.total_emitted - row.unmatched) / row.total_emitted).toFixed(4) : 0, + avg_resolution_rate: totalOutcomes > 0 ? +(resolvedCount / totalOutcomes).toFixed(4) : 0, + }; + }); +} + +/** + * Rule performance — **reporting view.** + * + * Mirror of `ruleScores()` (case-base.js) but intended for dashboards and + * reports, not promotion decisions: + * - Default threshold 1 (not 5): surface every rule that fired at least + * once so operators can see the long tail, including brand-new rules. + * - Includes `${check}.unmatched` fallback rule_ids (set by the pipeline + * for rule-less diagnostics — A4). These don't correspond to a + * registered rule, but they belong in reporting so the coverage gap is + * visible. + * - Omits the `disabled` flag. Disabling is a promotion decision; reports + * shouldn't imply one by displaying a derived threshold label. + * - Uses `EXISTS` for the outcome join. Post-A1 `outcomes` carries one row + * per (session, file, fp); a plain JOIN cross-multiplies with per-emit + * diagnostic rows. EXISTS keeps counts at one-per-outcome. + * + * @param {object} store + * @param {object} [opts] + * @param {number} [opts.minEmitted=1] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array} + */ +export function rulePerformance(store, { minEmitted = 1, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const ruleRows = store.query(` + SELECT d.hint_rule_id as rule_id, + COUNT(*) as emitted, + MIN(d.check_name) as check_name + FROM diagnostics d + WHERE d.hint_rule_id IS NOT NULL + AND d.hint_rule_id != 'unknown' + AND d.suppressed = 0 + ${sinceAnd} + GROUP BY d.hint_rule_id + HAVING COUNT(*) >= ? + `, [...sinceP, minEmitted]); + + const scores = []; + + for (const row of ruleRows) { + const outcomeRows = store.query(` + SELECT o.outcome, o.fix_applied, COUNT(*) as cnt + FROM outcomes o + WHERE EXISTS ( + SELECT 1 FROM diagnostics d + WHERE d.fp = o.fp AND d.hint_rule_id = ? AND d.suppressed = 0 ${sinceAnd} + ) + GROUP BY o.outcome, o.fix_applied + `, [row.rule_id, ...sinceP]); + + let resolved = 0, regressed = 0, unchanged = 0, moved = 0; + let adopted = 0, totalOutcomes = 0; + + for (const o of outcomeRows) { + totalOutcomes += o.cnt; + if (o.outcome === 'resolved') resolved += o.cnt; + else if (o.outcome === 'regressed') regressed += o.cnt; + else if (o.outcome === 'unchanged') unchanged += o.cnt; + else if (o.outcome === 'moved') moved += o.cnt; + if (o.fix_applied === 'verbatim') adopted += o.cnt; + } + + const resolutionRate = totalOutcomes > 0 ? resolved / totalOutcomes : 0; + const regressionRate = totalOutcomes > 0 ? regressed / totalOutcomes : 0; + const adoptionRate = totalOutcomes > 0 ? adopted / totalOutcomes : 0; + const effectiveness = resolutionRate - regressionRate; + + scores.push({ + rule_id: row.rule_id, + check: row.check_name, + emitted: row.emitted, + total_outcomes: totalOutcomes, + resolved, + regressed, + unchanged, + moved, + adopted, + resolution_rate: resolutionRate, + regression_rate: regressionRate, + adoption_rate: adoptionRate, + effectiveness, + unmatched: row.rule_id.endsWith('.unmatched'), + }); + } + + scores.sort((a, b) => a.effectiveness - b.effectiveness); + return scores; +} + +/** + * Rule drilldown — detailed diagnostic samples for a specific rule. + * Returns recent instances where this rule fired, with outcomes, fix status, + * and file distribution. Used by the dashboard drill-down panel. + * + * @param {object} store + * @param {string} ruleId + * @param {object} [opts] + * @param {number} [opts.limit=30] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + */ +export function ruleDrilldown(store, ruleId, { limit = 30, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAndD = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceAndD2 = sinceTs ? 'AND d2.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + // Each diagnostic row gets at most one outcome via correlated subqueries, + // avoiding the cartesian product from a plain LEFT JOIN on fp. + const samples = store.query(` + SELECT d.rowid as did, d.fp, d.template_fp, d.file, d.check_name, d.ts, + d.confidence, d.session_id, d.content_hash, d.hint_md_hash, + (SELECT o.outcome FROM outcomes o WHERE o.fp = d.fp AND o.session_id = d.session_id ORDER BY o.id DESC LIMIT 1) as outcome, + (SELECT o.fix_applied FROM outcomes o WHERE o.fp = d.fp AND o.session_id = d.session_id ORDER BY o.id DESC LIMIT 1) as fix_applied, + (SELECT o.collateral_added FROM outcomes o WHERE o.fp = d.fp AND o.session_id = d.session_id ORDER BY o.id DESC LIMIT 1) as collateral_added, + (SELECT pf.new_text_hash FROM proposed_fixes pf WHERE pf.fp = d.fp AND pf.session_id = d.session_id LIMIT 1) as fix_hash, + (SELECT pf.range_json FROM proposed_fixes pf WHERE pf.fp = d.fp AND pf.session_id = d.session_id LIMIT 1) as fix_range_json, + (SELECT pf.rule_id FROM proposed_fixes pf WHERE pf.fp = d.fp AND pf.session_id = d.session_id LIMIT 1) as fix_rule_id + FROM diagnostics d + WHERE d.hint_rule_id = ? AND d.suppressed = 0 ${sinceAndD} + GROUP BY d.fp, d.session_id + ORDER BY d.ts DESC + LIMIT ? + `, [ruleId, ...sinceP, limit]); + + // File stats: count distinct fps per file, then count outcomes per fp (not per join row). + // Inner subqueries also filter by `since` so file-resolved/regressed counts stay + // consistent with the outer file emit count when the operator narrows the window. + const fileStats = store.query(` + SELECT d.file, + COUNT(DISTINCT d.fp) as emitted, + (SELECT COUNT(DISTINCT o.fp) FROM outcomes o + JOIN diagnostics d2 ON o.fp = d2.fp + WHERE d2.hint_rule_id = ? AND d2.file = d.file AND o.outcome = 'resolved' ${sinceAndD2}) as resolved, + (SELECT COUNT(DISTINCT o.fp) FROM outcomes o + JOIN diagnostics d2 ON o.fp = d2.fp + WHERE d2.hint_rule_id = ? AND d2.file = d.file AND o.outcome = 'regressed' ${sinceAndD2}) as regressed + FROM diagnostics d + WHERE d.hint_rule_id = ? AND d.suppressed = 0 ${sinceAndD} + GROUP BY d.file + ORDER BY emitted DESC + LIMIT 10 + `, [ruleId, ...sinceP, ruleId, ...sinceP, ruleId, ...sinceP]); + + const templateStats = store.query(` + SELECT d.template_fp, + COUNT(DISTINCT d.fp) as count, + (SELECT COUNT(DISTINCT o.fp) FROM outcomes o + JOIN diagnostics d2 ON o.fp = d2.fp + WHERE d2.hint_rule_id = ? AND d2.template_fp = d.template_fp AND o.outcome = 'resolved' ${sinceAndD2}) as resolved, + (SELECT COUNT(DISTINCT o.fp) FROM outcomes o + JOIN diagnostics d2 ON o.fp = d2.fp + WHERE d2.hint_rule_id = ? AND d2.template_fp = d.template_fp AND o.outcome = 'regressed' ${sinceAndD2}) as regressed, + MIN(d.file) as sample_file + FROM diagnostics d + WHERE d.hint_rule_id = ? AND d.suppressed = 0 ${sinceAndD} + GROUP BY d.template_fp + ORDER BY count DESC + LIMIT 10 + `, [ruleId, ...sinceP, ruleId, ...sinceP, ruleId, ...sinceP]); + + return { + rule_id: ruleId, + samples: samples.map(s => ({ + fp: s.fp, + template_fp: s.template_fp, + file: s.file, + check: s.check_name, + ts: s.ts, + confidence: s.confidence, + session_id: s.session_id, + outcome: s.outcome ?? null, + fix_applied: s.fix_applied ?? null, + collateral: s.collateral_added ?? 0, + content_hash: s.content_hash ?? null, + hint_md_hash: s.hint_md_hash ?? null, + fix_hash: s.fix_hash ?? null, + fix_range: tryParseJson(s.fix_range_json), + fix_rule_id: s.fix_rule_id ?? null, + })), + file_distribution: fileStats.map(f => ({ + file: f.file, + emitted: f.emitted, + resolved: f.resolved, + regressed: f.regressed, + })), + template_patterns: templateStats.map(t => ({ + template_fp: t.template_fp, + count: t.count, + resolved: t.resolved, + regressed: t.regressed, + sample_file: t.sample_file, + })), + }; +} + +/** + * I1 — heuristic + rule fix-proposal performance. + * + * Reports per-rule_id stats grouped by `proposed_fixes.rule_id`. Covers BOTH + * rule-engine rules (e.g. "UnknownFilter.suggest_nearest") and heuristic- + * generator variants (e.g. "heuristic:UnknownFilter.text_edit"). Complements + * `rulePerformance()`, which groups on `diagnostics.hint_rule_id` — that one + * measures rule *matching*, this one measures fix *proposal / adoption*. + * + * Uses EXISTS when joining back to diagnostics so post-A1 outcome dedup isn't + * re-inflated by multiple emit rows sharing the same fp. + * + * @param {object} store + * @param {object} [opts] + * @param {number} [opts.minProposed=1] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array<{ + * rule_id, source, fix_kind, + * proposed, outcomes, adopted_verbatim, adopted_partial, + * adoption_rate, resolution_rate, + * }>} + */ +export function fixRulePerformance(store, { minProposed = 1, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND pf.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const rows = store.query(` + SELECT pf.rule_id, + MIN(pf.kind) AS fix_kind, + COUNT(*) AS proposed + FROM proposed_fixes pf + WHERE pf.rule_id IS NOT NULL ${sinceAnd} + GROUP BY pf.rule_id + HAVING COUNT(*) >= ? + `, [...sinceP, minProposed]); + + const out = []; + for (const r of rows) { + const outcomeRows = store.query(` + SELECT o.outcome, o.fix_applied, COUNT(*) AS n + FROM outcomes o + WHERE EXISTS ( + SELECT 1 FROM proposed_fixes pf + WHERE pf.fp = o.fp AND pf.session_id = o.session_id AND pf.rule_id = ? ${sinceAnd} + ) + GROUP BY o.outcome, o.fix_applied + `, [r.rule_id, ...sinceP]); + + let resolved = 0, regressed = 0, unchanged = 0, moved = 0; + let adopted_verbatim = 0, adopted_partial = 0, adopted_none = 0, total = 0; + for (const o of outcomeRows) { + total += o.n; + if (o.outcome === 'resolved') resolved += o.n; + else if (o.outcome === 'regressed') regressed += o.n; + else if (o.outcome === 'unchanged') unchanged += o.n; + else if (o.outcome === 'moved') moved += o.n; + + if (o.fix_applied === 'verbatim') adopted_verbatim += o.n; + else if (o.fix_applied === 'partial') adopted_partial += o.n; + else adopted_none += o.n; + } + + const source = r.rule_id.startsWith('heuristic:') ? 'heuristic' : 'rule'; + out.push({ + rule_id: r.rule_id, + source, + fix_kind: r.fix_kind, + proposed: r.proposed, + outcomes: total, + resolved, + regressed, + unchanged, + moved, + adopted_verbatim, + adopted_partial, + adopted_none, + adoption_rate: total ? (adopted_verbatim + adopted_partial) / total : 0, + resolution_rate: total ? resolved / total : 0, + }); + } + + out.sort((a, b) => b.proposed - a.proposed); + return out; +} + +/** + * Part G — adaptive-mode impact summary. + * + * Answers "what is adaptive mode actually doing right now, and what would + * static mode have done differently?" Two halves: + * + * 1. Current adaptive state: + * - which rules are disabled (+ their scores & outcome counts) + * - active force-enable / force-disable overrides + * - avg |adaptive_confidence - raw_confidence| over the last N emits + * (measures how aggressively case-base is bending scores) + * + * 2. Counterfactual for the recent window: + * - diagnostics suppressed by auto-disable (would have been surfaced + * under static mode) + * - fix proposals contributed by promoted rules (would be missing under + * static mode — promoted rules only exist in adaptive) + * - net delta headline for the dashboard summary row + * + * The query itself is schema-only — it doesn't touch the engine. The caller + * (server.js → /api/engine/impact) merges in the live engine state + * (`getDisabledRuleDetails`, override sets, `listPromotedRules`) that isn't + * in the DB. + * + * @param {object} store + * @param {object} [opts] + * @param {number} [opts.windowMs=86400000] Look-back window, default 24h. + * @returns {{ + * window_ms, window_start, window_end, + * emits_in_window, rule_matched_in_window, + * avg_confidence_delta, + * confidence_delta_samples, + * suppressed_by_disabled, + * promoted_fix_contributions + * }} + */ +export function adaptiveModeImpact(store, { windowMs = 86400000 } = {}) { + const windowEnd = new Date().toISOString(); + const windowStart = new Date(Date.now() - windowMs).toISOString(); + + const emitsRow = store.queryOne(` + SELECT COUNT(*) AS cnt + FROM diagnostics + WHERE ts BETWEEN ? AND ? AND suppressed = 0 + `, [windowStart, windowEnd]); + + const ruleMatchedRow = store.queryOne(` + SELECT COUNT(*) AS cnt + FROM diagnostics + WHERE ts BETWEEN ? AND ? + AND suppressed = 0 + AND hint_rule_id IS NOT NULL + AND hint_rule_id != 'unknown' + AND hint_rule_id NOT LIKE '%.unmatched' + `, [windowStart, windowEnd]); + + // avg |adaptive - raw| is not computable from the DB — the store only + // records the final confidence. To surface *some* adjustment signal we + // return the spread of confidence values across rule-matched diagnostics + // in the window, which moves when case-base scoring bends confidences. + const confRow = store.queryOne(` + SELECT COUNT(*) AS n, + AVG(confidence) AS mean, + MIN(confidence) AS min_c, + MAX(confidence) AS max_c + FROM diagnostics + WHERE ts BETWEEN ? AND ? + AND suppressed = 0 + AND confidence IS NOT NULL + AND hint_rule_id NOT LIKE '%.unmatched' + `, [windowStart, windowEnd]); + + // Counterfactual: diagnostics whose hint_rule_id is in the currently- + // disabled set. The query can't know the live disabled set — it's set + // from the engine at call time — so we return a helper sub-query result + // keyed by rule_id that the caller filters. + const byRuleRows = store.query(` + SELECT hint_rule_id AS rule_id, COUNT(*) AS n + FROM diagnostics + WHERE ts BETWEEN ? AND ? AND suppressed = 0 + GROUP BY hint_rule_id + `, [windowStart, windowEnd]); + const byRule = {}; + for (const r of byRuleRows) if (r.rule_id) byRule[r.rule_id] = r.n; + + return { + window_ms: windowMs, + window_start: windowStart, + window_end: windowEnd, + emits_in_window: emitsRow?.cnt ?? 0, + rule_matched_in_window: ruleMatchedRow?.cnt ?? 0, + confidence: { + samples: confRow?.n ?? 0, + mean: confRow?.mean ?? null, + min: confRow?.min_c ?? null, + max: confRow?.max_c ?? null, + }, + // The caller (HTTP handler) takes this map + the live disabled set and + // sums up hits for only those rule_ids. Keeping the split here — DB + // side can't see the live engine state — means the query stays pure. + emits_by_rule: byRule, + }; +} diff --git a/src/core/analytics-store.js b/src/core/analytics-store.js new file mode 100644 index 0000000..e5e1d80 --- /dev/null +++ b/src/core/analytics-store.js @@ -0,0 +1,673 @@ +/** + * SQLite analytics cache — derived from session NDJSON event logs. + * + * The DB at `.pos-supervisor/analytics.db` is disposable: it can be + * rebuilt from event logs at any time via `rebuild()`. Never write + * truth here — only derived data. + * + * Uses WAL mode for concurrent reads during long writes (rebuild). + */ + +import { existsSync, readdirSync } from 'node:fs'; +import { join, dirname } from 'node:path'; +import { mkdirSync } from 'node:fs'; +import { readEventLog } from './session-events.js'; +import { classifySession, computeCollateral, buildEmitIndex, classifyFixAdoption } from './window-classifier.js'; + +let _Database = null; +function getDatabase() { + if (!_Database) { + try { + _Database = require('bun:sqlite').Database; + } catch { + throw new Error('bun:sqlite not available — analytics store requires Bun runtime'); + } + } + return _Database; +} + +const SCHEMA_VERSION = 6; + +const SCHEMA_SQL = ` + CREATE TABLE IF NOT EXISTS meta ( + key TEXT PRIMARY KEY, + value TEXT + ); + + CREATE TABLE IF NOT EXISTS events ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + session_id TEXT NOT NULL, + kind TEXT NOT NULL, + ts TEXT NOT NULL, + payload TEXT NOT NULL + ); + CREATE INDEX IF NOT EXISTS idx_events_session ON events(session_id); + CREATE INDEX IF NOT EXISTS idx_events_kind ON events(kind); + CREATE INDEX IF NOT EXISTS idx_events_ts ON events(ts); + + CREATE TABLE IF NOT EXISTS diagnostics ( + fp TEXT NOT NULL, + template_fp TEXT, + session_id TEXT NOT NULL, + file TEXT NOT NULL, + check_name TEXT NOT NULL, + severity TEXT, + ts TEXT NOT NULL, + hint_rule_id TEXT, + hint_md_hash TEXT, + content_hash TEXT, + suppressed INTEGER DEFAULT 0, + confidence REAL + ); + CREATE INDEX IF NOT EXISTS idx_diag_fp ON diagnostics(fp); + CREATE INDEX IF NOT EXISTS idx_diag_session ON diagnostics(session_id); + CREATE INDEX IF NOT EXISTS idx_diag_check ON diagnostics(check_name); + CREATE INDEX IF NOT EXISTS idx_diag_file ON diagnostics(file); + -- One diagnostic row per (session, file, fp). Re-validations of the same + -- file in the same session are ignored — first emit is the canonical one. + -- See migrate_v2_to_v3 for the upgrade path on existing DBs. + CREATE UNIQUE INDEX IF NOT EXISTS idx_diag_sff ON diagnostics(session_id, file, fp); + + CREATE TABLE IF NOT EXISTS proposed_fixes ( + fp TEXT NOT NULL, + session_id TEXT NOT NULL, + ts TEXT NOT NULL, + range_json TEXT, + new_text_hash TEXT, + kind TEXT, + rule_id TEXT + ); + CREATE INDEX IF NOT EXISTS idx_fixes_fp ON proposed_fixes(fp); + CREATE INDEX IF NOT EXISTS idx_fixes_rule ON proposed_fixes(rule_id); + + CREATE TABLE IF NOT EXISTS windows ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + session_id TEXT NOT NULL, + file TEXT NOT NULL, + idx INTEGER NOT NULL, + ts_start TEXT NOT NULL, + ts_end TEXT NOT NULL, + content_hash_start TEXT, + content_hash_end TEXT, + is_draft INTEGER DEFAULT 0, + closed_by TEXT DEFAULT 'validate' + ); + CREATE INDEX IF NOT EXISTS idx_windows_session ON windows(session_id); + CREATE INDEX IF NOT EXISTS idx_windows_file ON windows(session_id, file); + + CREATE TABLE IF NOT EXISTS outcomes ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + fp TEXT NOT NULL, + window_id INTEGER NOT NULL REFERENCES windows(id), + outcome TEXT NOT NULL, + fix_applied TEXT, + collateral_added INTEGER DEFAULT 0, + session_id TEXT, + file TEXT + ); + CREATE INDEX IF NOT EXISTS idx_outcomes_fp ON outcomes(fp); + CREATE INDEX IF NOT EXISTS idx_outcomes_window ON outcomes(window_id); + -- One outcome per (session, file, fp). Terminal state wins via INSERT OR REPLACE. + -- See migrate_v1_to_v2 for the upgrade path on existing DBs. + CREATE UNIQUE INDEX IF NOT EXISTS idx_outcomes_sff ON outcomes(session_id, file, fp); + + CREATE TABLE IF NOT EXISTS rule_promotions ( + rule_id TEXT PRIMARY KEY, + check_name TEXT NOT NULL, + template_fp TEXT NOT NULL, + promoted_at TEXT NOT NULL, + probation INTEGER DEFAULT 1, + resolved_at TEXT, + resolution TEXT + ); + + CREATE TABLE IF NOT EXISTS health_scores ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + ts TEXT NOT NULL, + score INTEGER NOT NULL, + mode TEXT NOT NULL, + dimensions TEXT NOT NULL + ); + CREATE INDEX IF NOT EXISTS idx_health_ts ON health_scores(ts); +`; + +export function openAnalyticsStore(dbPath, { readonly = false, blobStore = null } = {}) { + if (!dbPath) throw new Error('openAnalyticsStore: dbPath required'); + mkdirSync(dirname(dbPath), { recursive: true }); + + const Database = getDatabase(); + const dbOpts = readonly ? { readonly: true } : { create: true, readwrite: true }; + const db = new Database(dbPath, dbOpts); + if (!readonly) { + db.exec('PRAGMA journal_mode = WAL'); + db.exec('PRAGMA synchronous = NORMAL'); + db.exec('PRAGMA foreign_keys = ON'); + // Migrations run BEFORE SCHEMA_SQL so existing tables can be reshaped + // without tripping IF-NOT-EXISTS guards (e.g. adding a UNIQUE INDEX + // over already-duplicated rows would fail if SCHEMA_SQL ran first). + migrate(db); + db.exec(SCHEMA_SQL); + setMeta(db, 'schema_version', String(SCHEMA_VERSION)); + } + + const stmts = readonly ? {} : prepareStatements(db); + + function close() { + try { db.close(); } catch {} + } + + function ingestEvent(event) { + const { v, session_id, ts, kind, ...payload } = event; + stmts.insertEvent.run(session_id, kind, ts, JSON.stringify(payload)); + + if (kind === 'validator_emit') { + ingestValidatorEmit(event, stmts); + } + } + + function ingestSession(sessionDir) { + const eventsPath = join(sessionDir, 'events.ndjson'); + if (!existsSync(eventsPath)) return 0; + const events = readEventLog(eventsPath); + db.exec('BEGIN'); + try { + for (const event of events) { + ingestEvent(event); + } + classifyAndStoreWindows(events); + db.exec('COMMIT'); + return events.length; + } catch (e) { + db.exec('ROLLBACK'); + throw e; + } + } + + function classifyAndStoreWindows(events) { + const windowResults = classifySession(events); + // Per-fp index of validator_emit events → we pull proposed_fixes off the + // latest emit in the window's time range below. Built once per session. + const emitIndex = buildEmitIndex(events); + + for (const { window: w, outcomes } of windowResults) { + const collateral = computeCollateral(outcomes); + const windowId = insertWindow(w); + + // Resolve window content blobs once per window. classifyFixAdoption is + // a hot-ish path inside rebuilds — avoid re-reading per outcome. + const startContent = blobStore && w.content_hash_start + ? blobStore.getText(w.content_hash_start) : null; + const endContent = blobStore && w.content_hash_end + ? blobStore.getText(w.content_hash_end) : null; + + for (const o of outcomes) { + insertOutcome({ + fp: o.fp, + window_id: windowId, + outcome: o.outcome, + fix_applied: classifyOutcomeFixAdoption( + o, w, emitIndex, startContent, endContent, + ), + collateral_added: o.outcome === 'regressed' ? collateral : 0, + session_id: w.session_id, + file: w.file, + }); + } + } + } + + /** + * Map a window outcome to a fix_applied label ('verbatim' | 'partial' | + * 'ignored' | null). Skipped for: + * - regressed — the fp is *new* at window end, so there's no "fix that + * was proposed before and then applied"; classification is meaningless. + * - write_unverified — no end state to compare against. + * - missing content blobs — can't reconstruct start/end text. + * - no proposed fix in the window — no adoption to classify. + */ + function classifyOutcomeFixAdoption(outcome, w, emitIndex, startContent, endContent) { + if (!blobStore) return null; + if (outcome.outcome === 'regressed') return null; + if (outcome.outcome === 'write_unverified') return null; + if (!startContent || !endContent) return null; + + // Pick the emit that the agent actually saw at window start — that's the + // proposal-set the agent could have adopted. Falling back to "latest at or + // before ts_end" is wrong: it can capture a newer proposal the agent never + // saw (e.g. a rule whose `apply()` output changed between re-validations). + const emits = emitIndex.get(outcome.fp) ?? []; + let chosen = null; + for (const e of emits) { + if (e.session_id !== w.session_id) continue; + if (e.file !== w.file) continue; + if (e.ts > w.ts_start) continue; + if (!chosen || e.ts > chosen.ts) chosen = e; + } + // If no emit at or before ts_start exists, fall back to the earliest in + // the window — this covers ingestion paths where the emit and the + // tool_call share a timestamp and strict `<=` excludes the right row. + if (!chosen) { + for (const e of emits) { + if (e.session_id !== w.session_id) continue; + if (e.file !== w.file) continue; + if (e.ts > w.ts_end) continue; + if (!chosen || e.ts < chosen.ts) chosen = e; + } + } + const fixes = chosen?.proposed_fixes ?? []; + if (fixes.length === 0) return null; + + return classifyFixAdoption(startContent, endContent, fixes, blobStore); + } + + function rebuild(sessionsDir) { + if (!existsSync(sessionsDir)) return { sessions: 0, events: 0 }; + db.exec('BEGIN'); + try { + db.exec('DELETE FROM outcomes'); + db.exec('DELETE FROM windows'); + db.exec('DELETE FROM proposed_fixes'); + db.exec('DELETE FROM diagnostics'); + db.exec('DELETE FROM events'); + db.exec('COMMIT'); + } catch (e) { + db.exec('ROLLBACK'); + throw e; + } + + let totalEvents = 0; + let totalSessions = 0; + const entries = readdirSync(sessionsDir, { withFileTypes: true }); + for (const entry of entries) { + if (!entry.isDirectory()) continue; + const sessionDir = join(sessionsDir, entry.name); + const count = ingestSession(sessionDir); + if (count > 0) totalSessions++; + totalEvents += count; + } + setMeta(db, 'last_rebuild', new Date().toISOString()); + return { sessions: totalSessions, events: totalEvents }; + } + + function insertWindow(row) { + return stmts.insertWindow.run( + row.session_id, row.file, row.idx, + row.ts_start, row.ts_end, + row.content_hash_start ?? null, + row.content_hash_end ?? null, + row.is_draft ? 1 : 0, + row.closed_by ?? 'validate', + ).lastInsertRowid; + } + + function insertOutcome(row) { + let session_id = row.session_id ?? null; + let file = row.file ?? null; + if ((!session_id || !file) && row.window_id != null) { + const w = stmts.selectWindowById.get(row.window_id); + if (w) { + session_id = session_id || w.session_id; + file = file || w.file; + } + } + stmts.insertOutcome.run( + row.fp, row.window_id, row.outcome, + row.fix_applied ?? null, + row.collateral_added ?? 0, + session_id, + file, + ); + } + + function query(sql, params = []) { + return db.prepare(sql).all(...params); + } + + function queryOne(sql, params = []) { + return db.prepare(sql).get(...params); + } + + function getMeta(key) { + const row = db.prepare('SELECT value FROM meta WHERE key = ?').get(key); + return row ? row.value : null; + } + + /** + * Reporting-window baseline. Operator-set checkpoint that filters every + * dashboard / Markdown-report query to "stats from this timestamp forward". + * Engine-state callers (case-base.ruleScores for syncDisabledRules, + * scoreRule, cac-predictor history providers, server-status disabledRules) + * MUST keep using full history — they pass `since: null` explicitly. Reporting + * callers that omit `since` resolve it via this helper. + * + * Two meta keys: `analytics_baseline_ts` (the timestamp itself) and + * `analytics_baseline_set_at` (when the operator set it — audit trail). + * Both absent ⇒ no baseline ⇒ full history (default behaviour, identical + * to pre-baseline releases). + */ + function getBaselineTs() { + return getMeta('analytics_baseline_ts'); + } + + function getBaselineMeta() { + return { + baseline_ts: getMeta('analytics_baseline_ts'), + set_at: getMeta('analytics_baseline_set_at'), + }; + } + + function setBaselineTs(iso) { + if (iso === null || iso === undefined) { + clearBaseline(); + return; + } + if (typeof iso !== 'string' || iso.length === 0) { + throw new TypeError(`setBaselineTs: expected ISO string or null, got ${typeof iso}`); + } + const parsed = new Date(iso); + if (Number.isNaN(parsed.getTime())) { + throw new TypeError(`setBaselineTs: '${iso}' is not a valid date`); + } + setMeta(db, 'analytics_baseline_ts', iso); + setMeta(db, 'analytics_baseline_set_at', new Date().toISOString()); + } + + function clearBaseline() { + db.prepare('DELETE FROM meta WHERE key IN (?, ?)').run( + 'analytics_baseline_ts', 'analytics_baseline_set_at', + ); + } + + function stats() { + const events = db.prepare('SELECT COUNT(*) as count FROM events').get().count; + const diagnostics = db.prepare('SELECT COUNT(*) as count FROM diagnostics').get().count; + const windows = db.prepare('SELECT COUNT(*) as count FROM windows').get().count; + const outcomes = db.prepare('SELECT COUNT(*) as count FROM outcomes').get().count; + const sessions = db.prepare('SELECT COUNT(DISTINCT session_id) as count FROM events').get().count; + return { events, diagnostics, windows, outcomes, sessions, schema_version: SCHEMA_VERSION }; + } + + function recordPromotion({ rule_id, check_name, template_fp }) { + stmts.insertPromotion.run( + rule_id, check_name, template_fp, + new Date().toISOString(), 1, null, null, + ); + } + + function resolvePromotion(rule_id, resolution) { + db.prepare( + `UPDATE rule_promotions SET probation = 0, resolved_at = ?, resolution = ? WHERE rule_id = ?`, + ).run(new Date().toISOString(), resolution, rule_id); + } + + function getPromotion(rule_id) { + return db.prepare('SELECT * FROM rule_promotions WHERE rule_id = ?').get(rule_id) ?? null; + } + + function getPromotionsOnProbation() { + return db.prepare('SELECT * FROM rule_promotions WHERE probation = 1').all(); + } + + function insertHealthScore({ score, mode, dimensions }) { + stmts.insertHealthScore.run( + new Date().toISOString(), score, mode, + JSON.stringify(dimensions), + ); + } + + function getHealthScores({ limit = 30, mode } = {}) { + const sql = mode + ? 'SELECT * FROM health_scores WHERE mode = ? ORDER BY ts DESC LIMIT ?' + : 'SELECT * FROM health_scores ORDER BY ts DESC LIMIT ?'; + const params = mode ? [mode, limit] : [limit]; + const rows = db.prepare(sql).all(...params); + return rows.reverse().map(r => ({ + ...r, + dimensions: JSON.parse(r.dimensions), + })); + } + + return { + close, + ingestEvent, + ingestSession, + rebuild, + insertWindow, + insertOutcome, + query, + queryOne, + getMeta, + getBaselineTs, + getBaselineMeta, + setBaselineTs, + clearBaseline, + stats, + recordPromotion, + resolvePromotion, + getPromotion, + getPromotionsOnProbation, + insertHealthScore, + getHealthScores, + get db() { return db; }, + get path() { return dbPath; }, + }; +} + +function prepareStatements(db) { + return { + insertEvent: db.prepare( + 'INSERT INTO events (session_id, kind, ts, payload) VALUES (?, ?, ?, ?)', + ), + insertDiag: db.prepare( + `INSERT OR IGNORE INTO diagnostics (fp, template_fp, session_id, file, check_name, severity, ts, hint_rule_id, hint_md_hash, content_hash, suppressed, confidence) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ), + insertFix: db.prepare( + `INSERT INTO proposed_fixes (fp, session_id, ts, range_json, new_text_hash, kind, rule_id) + VALUES (?, ?, ?, ?, ?, ?, ?)`, + ), + insertWindow: db.prepare( + `INSERT INTO windows (session_id, file, idx, ts_start, ts_end, content_hash_start, content_hash_end, is_draft, closed_by) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ), + // INSERT OR REPLACE + UNIQUE(session_id, file, fp) is the dedup mechanism: + // as classifySession walks windows chronologically, each replace stamps + // the terminal (session, file, fp) classification. + insertOutcome: db.prepare( + `INSERT OR REPLACE INTO outcomes + (fp, window_id, outcome, fix_applied, collateral_added, session_id, file) + VALUES (?, ?, ?, ?, ?, ?, ?)`, + ), + selectWindowById: db.prepare( + `SELECT session_id, file FROM windows WHERE id = ?`, + ), + insertPromotion: db.prepare( + `INSERT OR REPLACE INTO rule_promotions (rule_id, check_name, template_fp, promoted_at, probation, resolved_at, resolution) + VALUES (?, ?, ?, ?, ?, ?, ?)`, + ), + insertHealthScore: db.prepare( + `INSERT INTO health_scores (ts, score, mode, dimensions) + VALUES (?, ?, ?, ?)`, + ), + }; +} + +function ingestValidatorEmit(event, stmts) { + stmts.insertDiag.run( + event.fp, + event.template_fp ?? null, + event.session_id, + event.file, + event.check ?? event.hint_rule_id ?? 'unknown', + null, + event.ts, + event.hint_rule_id ?? null, + event.hint_md_hash ?? null, + event.content_hash ?? null, + 0, + event.confidence ?? null, + ); + + if (event.proposed_fixes?.length) { + for (const fix of event.proposed_fixes) { + stmts.insertFix.run( + event.fp, + event.session_id, + event.ts, + fix.range ? JSON.stringify(fix.range) : null, + fix.new_text_hash ?? null, + fix.kind ?? null, + fix.rule_id ?? null, + ); + } + } +} + +function setMeta(db, key, value) { + db.prepare('INSERT OR REPLACE INTO meta (key, value) VALUES (?, ?)').run(key, value); +} + +/** + * Schema migrations. Called before SCHEMA_SQL so existing tables can be + * reshaped (e.g. adding columns, dedup, adding UNIQUE INDEX) without the + * CREATE … IF NOT EXISTS guards masking stale shape. + * + * New DBs skip migrations — meta.schema_version is absent (treated as 0) but + * the tables SCHEMA_SQL creates already match SCHEMA_VERSION. A fresh DB + * hits migrate_v1_to_v2 as a no-op because the outcomes table doesn't + * exist yet. Safe. + */ +function migrate(db) { + // Meta table must exist before we can read schema_version. + db.exec(`CREATE TABLE IF NOT EXISTS meta (key TEXT PRIMARY KEY, value TEXT)`); + const row = db.prepare('SELECT value FROM meta WHERE key = ?').get('schema_version'); + const current = row ? Number(row.value) : 0; + + if (current < 2) migrate_v1_to_v2(db); + if (current < 3) migrate_v2_to_v3(db); + if (current < 4) migrate_v3_to_v4(db); + if (current < 5) migrate_v4_to_v5(db); + if (current < 6) migrate_v5_to_v6(db); +} + +function migrate_v1_to_v2(db) { + // No-op for fresh DBs: the outcomes table doesn't exist yet. + const outcomesExists = db.prepare( + `SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'outcomes'`, + ).get(); + if (!outcomesExists) return; + + const cols = db.prepare(`PRAGMA table_info(outcomes)`).all().map(r => r.name); + if (!cols.includes('session_id')) { + db.exec(`ALTER TABLE outcomes ADD COLUMN session_id TEXT`); + } + if (!cols.includes('file')) { + db.exec(`ALTER TABLE outcomes ADD COLUMN file TEXT`); + } + + // Backfill from windows via window_id. Correlated subquery works across all + // SQLite versions (UPDATE FROM requires 3.33+). + db.exec(` + UPDATE outcomes + SET + session_id = COALESCE(session_id, (SELECT w.session_id FROM windows w WHERE w.id = outcomes.window_id)), + file = COALESCE(file, (SELECT w.file FROM windows w WHERE w.id = outcomes.window_id)) + WHERE session_id IS NULL OR file IS NULL + `); + + // Dedup: keep the highest-id row per (session_id, file, fp). classifySession + // walks windows chronologically, so the highest id is the terminal state. + // Rows with NULL session_id or file are orphans (missing window); drop them. + db.exec(` + DELETE FROM outcomes + WHERE session_id IS NULL OR file IS NULL + `); + db.exec(` + DELETE FROM outcomes + WHERE id NOT IN ( + SELECT MAX(id) FROM outcomes + GROUP BY session_id, file, fp + ) + `); + + // UNIQUE INDEX goes up after dedup; SCHEMA_SQL's IF-NOT-EXISTS guard then + // becomes a no-op on the next open. + db.exec(` + CREATE UNIQUE INDEX IF NOT EXISTS idx_outcomes_sff + ON outcomes(session_id, file, fp) + `); +} + +function migrate_v2_to_v3(db) { + // No-op for fresh DBs: the diagnostics table doesn't exist yet. + const diagExists = db.prepare( + `SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'diagnostics'`, + ).get(); + if (!diagExists) return; + + // Dedup: keep the earliest row per (session_id, file, fp). The first emit + // is canonical; later re-validations of the same file in the same session + // carry the same diagnostic and add no new information. + db.exec(` + DELETE FROM diagnostics + WHERE rowid NOT IN ( + SELECT MIN(rowid) FROM diagnostics + GROUP BY session_id, file, fp + ) + `); + + db.exec(` + CREATE UNIQUE INDEX IF NOT EXISTS idx_diag_sff + ON diagnostics(session_id, file, fp) + `); +} + +function migrate_v3_to_v4(db) { + // No-op for fresh DBs: windows table doesn't exist yet (created by SCHEMA_SQL). + const windowsExists = db.prepare( + `SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'windows'`, + ).get(); + if (!windowsExists) return; + + const cols = db.prepare('PRAGMA table_info(windows)').all().map(c => c.name); + if (!cols.includes('is_draft')) { + db.exec('ALTER TABLE windows ADD COLUMN is_draft INTEGER DEFAULT 0'); + } + if (!cols.includes('closed_by')) { + db.exec("ALTER TABLE windows ADD COLUMN closed_by TEXT DEFAULT 'validate'"); + } +} + +function migrate_v4_to_v5(db) { + // No-op for fresh DBs: diagnostics table doesn't exist yet. + const diagExists = db.prepare( + `SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'diagnostics'`, + ).get(); + if (!diagExists) return; + + const cols = db.prepare('PRAGMA table_info(diagnostics)').all().map(c => c.name); + if (!cols.includes('hint_md_hash')) { + db.exec('ALTER TABLE diagnostics ADD COLUMN hint_md_hash TEXT'); + } + // No backfill: the event log still carries `hint_md_hash` in each + // validator_emit payload — a subsequent store.rebuild(sessionsDir) replays + // it correctly into the new column. Migration alone just unblocks future + // writes; operators that want historical hints must run a rebuild. +} + +function migrate_v5_to_v6(db) { + // No-op for fresh DBs: proposed_fixes table doesn't exist yet. + const tblExists = db.prepare( + `SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'proposed_fixes'`, + ).get(); + if (!tblExists) return; + + const cols = db.prepare('PRAGMA table_info(proposed_fixes)').all().map(c => c.name); + if (!cols.includes('rule_id')) { + db.exec('ALTER TABLE proposed_fixes ADD COLUMN rule_id TEXT'); + } + db.exec('CREATE INDEX IF NOT EXISTS idx_fixes_rule ON proposed_fixes(rule_id)'); + // Existing rows stay at NULL rule_id (pre-I1 events don't carry attribution). + // A store.rebuild(sessionsDir) backfills from newer events; older events are + // genuinely un-attributed and will remain that way. +} diff --git a/src/core/blob-store.js b/src/core/blob-store.js new file mode 100644 index 0000000..ef8d401 --- /dev/null +++ b/src/core/blob-store.js @@ -0,0 +1,206 @@ +/** + * Content-addressed blob store under `.pos-supervisor/blobs/`. + * + * Why this exists (roadmap §A4): + * Phase A5's `validator_emit` event references "the content the validator + * actually saw" by `content_hash`. The hash is meaningless without a way + * to resolve it back to the bytes — that's this store. Analytics in + * Phase B reads (file_hash, fp) tuples and needs to recover the source + * to render "here's what was shown" panels in the dashboard. + * + * It is intentionally NOT a general cache. Two callers writing the same + * bytes get the same blob path (sha256). Eviction is LRU-by-atime so a + * long session can't blow disk past the configured cap. + * + * Layout: `///` — two-level sharding so a single + * directory never accumulates millions of entries (matters on ext4 / NTFS, + * not so much on bcachefs / APFS, but cheap insurance). + * + * Concurrency: writes are atomic via `.tmp.` → rename. Reads + * never lock. The whole store assumes one supervisor process per project + * (the lock model the rest of the supervisor uses). + * + * Failure model: every operation can throw — the caller decides whether + * a blob miss is fatal. Callers in the tool path should catch and degrade + * to "content not stored" rather than failing the validate_code response. + */ + +import { createHash } from 'node:crypto'; +import { + existsSync, mkdirSync, readFileSync, readdirSync, renameSync, + rmSync, statSync, writeFileSync, utimesSync, +} from 'node:fs'; +import { join } from 'node:path'; + +export const BLOB_HASH_ALGO = 'sha256'; +export const DEFAULT_MAX_BYTES = 100 * 1024 * 1024; // 100 MiB +export const DEFAULT_MAX_FILES = 10_000; + +/** + * Compute the content hash for a string or Buffer. Exposed so callers can + * derive a blob's address without writing it (e.g. fingerprint logging). + */ +export function blobHash(content) { + const h = createHash(BLOB_HASH_ALGO); + h.update(content); + return h.digest('hex'); +} + +/** + * Open a blob store rooted at `dir`. The directory is created on demand. + * + * @param {string} dir Root path (e.g. `/.pos-supervisor/blobs`). + * @param {object} [opts] + * @param {number} [opts.maxBytes=DEFAULT_MAX_BYTES] Hard cap on total stored bytes. + * @param {number} [opts.maxFiles=DEFAULT_MAX_FILES] Hard cap on stored file count. + * @returns {object} store API (see methods below) + */ +export function openBlobStore(dir, { maxBytes = DEFAULT_MAX_BYTES, maxFiles = DEFAULT_MAX_FILES } = {}) { + if (!dir) throw new Error('openBlobStore: dir required'); + mkdirSync(dir, { recursive: true }); + + function pathFor(hash) { + if (typeof hash !== 'string' || hash.length < 4) { + throw new Error('blob-store: hash must be a hex string of length >= 4'); + } + return join(dir, hash.slice(0, 2), hash.slice(2, 4), hash.slice(4)); + } + + function exists(hash) { + return existsSync(pathFor(hash)); + } + + /** + * Persist `content` and return its sha256 hash. Idempotent: re-writing + * the same content is a noop (and bumps atime so the LRU notices). + */ + function put(content) { + const hash = blobHash(content); + const target = pathFor(hash); + if (existsSync(target)) { + // Touch atime so this blob looks "fresh" to the LRU sweeper. + try { utimesSync(target, new Date(), statSync(target).mtime); } catch {} + return hash; + } + mkdirSync(join(dir, hash.slice(0, 2), hash.slice(2, 4)), { recursive: true }); + // Atomic write: stage to a sibling temp file, then rename. fs rename + // within the same dir is atomic on POSIX and Windows. + const tmp = `${target}.tmp.${process.pid}.${Math.random().toString(36).slice(2)}`; + writeFileSync(tmp, content); + renameSync(tmp, target); + enforceLimits(); + return hash; + } + + /** + * Read blob bytes. Returns Buffer or null if the hash isn't stored. + * On hit, atime is bumped so the LRU keeps it fresh. + */ + function get(hash) { + const p = pathFor(hash); + if (!existsSync(p)) return null; + try { + const data = readFileSync(p); + try { utimesSync(p, new Date(), statSync(p).mtime); } catch {} + return data; + } catch { + return null; + } + } + + /** Same as get(), but decoded as utf-8. Returns null on miss. */ + function getText(hash) { + const buf = get(hash); + return buf == null ? null : buf.toString('utf-8'); + } + + /** + * Remove a single blob. Returns true if the file existed and was removed. + * Empty shard directories are NOT cleaned up (they're tiny and become + * cache-friendly when the next blob lands in the same shard). + */ + function remove(hash) { + const p = pathFor(hash); + if (!existsSync(p)) return false; + try { rmSync(p, { force: true }); return true; } + catch { return false; } + } + + /** + * Walk the store and return every blob with its size + atime, ordered + * oldest-atime-first. Used by enforceLimits and stats. O(n) in stored + * blobs — fine for the cap sizes we target. + */ + function listEntries() { + const out = []; + if (!existsSync(dir)) return out; + for (const a of safeReaddir(dir)) { + const subA = join(dir, a); + if (!safeIsDir(subA)) continue; + for (const b of safeReaddir(subA)) { + const subB = join(subA, b); + if (!safeIsDir(subB)) continue; + for (const name of safeReaddir(subB)) { + // Skip in-flight temp writes from concurrent put()s. + if (name.includes('.tmp.')) continue; + const full = join(subB, name); + let st; + try { st = statSync(full); } catch { continue; } + if (!st.isFile()) continue; + out.push({ hash: a + b + name, path: full, size: st.size, atimeMs: st.atimeMs }); + } + } + } + out.sort((x, y) => x.atimeMs - y.atimeMs); + return out; + } + + /** Rollup stats — { count, bytes, maxBytes, maxFiles } — for the dashboard. */ + function stats() { + const entries = listEntries(); + let bytes = 0; + for (const e of entries) bytes += e.size; + return { count: entries.length, bytes, maxBytes, maxFiles }; + } + + /** + * Drop oldest blobs until we're under both caps. Called automatically + * after every put(); exposed for explicit sweeps (e.g. a shutdown hook). + * Returns the number of evicted blobs. + */ + function enforceLimits() { + const entries = listEntries(); + let bytes = entries.reduce((acc, e) => acc + e.size, 0); + let count = entries.length; + let evicted = 0; + let i = 0; + while ((bytes > maxBytes || count > maxFiles) && i < entries.length) { + const e = entries[i++]; + try { rmSync(e.path, { force: true }); } + catch { continue; } + bytes -= e.size; + count -= 1; + evicted += 1; + } + return evicted; + } + + return { + put, get, getText, exists, remove, + enforceLimits, stats, listEntries, + pathFor, + get root() { return dir; }, + get maxBytes() { return maxBytes; }, + get maxFiles() { return maxFiles; }, + }; +} + +// ── Internal helpers ───────────────────────────────────────────────────────── + +function safeReaddir(p) { + try { return readdirSync(p); } catch { return []; } +} + +function safeIsDir(p) { + try { return statSync(p).isDirectory(); } catch { return false; } +} diff --git a/src/core/cac-config.js b/src/core/cac-config.js new file mode 100644 index 0000000..6501e6d --- /dev/null +++ b/src/core/cac-config.js @@ -0,0 +1,119 @@ +/** + * CAC predictor configuration — persisted at + * `/.pos-supervisor/cac-config.json`. + * + * The CAC (Cohen's Agentic Conjecture) layer is an OPT-IN 4th gating axis + * applied to diagnostics after the existing cascade + * (severity → static confidence → adaptive-mode → force-disable). It uses + * historical adoption data from the analytics store to predict the probability + * that an agent will adopt the proposed fix for a given diagnostic, and + * suppresses or downgrades emits whose predicted adoption falls below the + * configured threshold. + * + * Defaults: DISABLED. The validator behaves identically to versions that + * predate this module until an operator explicitly turns it on from the + * dashboard. Even when enabled, the default mode is `shadow`, which records + * decisions to the session bus but does not modify diagnostics. + * + * File schema (JSON): + * { + * "version": 1, + * "enabled": false, + * "mode": "shadow" | "active", + * "threshold": 0.30, + * "action": "downgrade" | "suppress", + * "min_samples": 5 + * } + * + * Reads are tolerant — a missing file or malformed JSON yields the safe + * default state (disabled). Writes are atomic (temp + rename) so a crash + * mid-save can't leave the file half-written. Mirrors `rule-overrides.js`. + */ + +import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; + +const FILE_VERSION = 1; +const FILE_NAME = 'cac-config.json'; + +export const VALID_MODES = ['shadow', 'active']; +export const VALID_ACTIONS = ['downgrade', 'suppress']; + +function configPath(projectDir) { + return join(projectDir, '.pos-supervisor', FILE_NAME); +} + +export function defaultCacConfig() { + return { + version: FILE_VERSION, + enabled: false, + mode: 'shadow', + threshold: 0.30, + action: 'downgrade', + min_samples: 5, + }; +} + +function coerceConfig(raw) { + const def = defaultCacConfig(); + const out = { ...def }; + if (typeof raw?.enabled === 'boolean') out.enabled = raw.enabled; + if (VALID_MODES.includes(raw?.mode)) out.mode = raw.mode; + if (VALID_ACTIONS.includes(raw?.action)) out.action = raw.action; + if (typeof raw?.threshold === 'number' && raw.threshold >= 0 && raw.threshold <= 1) { + out.threshold = raw.threshold; + } + if (Number.isInteger(raw?.min_samples) && raw.min_samples >= 0) { + out.min_samples = raw.min_samples; + } + return out; +} + +/** + * Load config from disk. Never throws — on any error returns default state + * and calls `log` if provided. A corrupt config file must not prevent the + * server from starting or stop validate_code from running. + */ +export function loadCacConfig(projectDir, { log } = {}) { + const path = configPath(projectDir); + if (!existsSync(path)) return defaultCacConfig(); + try { + const raw = readFileSync(path, 'utf-8'); + const parsed = JSON.parse(raw); + return coerceConfig(parsed); + } catch (e) { + log?.(`cac-config: failed to parse ${path} (${e.message}); using defaults`); + return defaultCacConfig(); + } +} + +/** + * Atomic write: stage to a sibling temp file, then rename. fs rename within + * the same dir is atomic on POSIX. A reader during the write sees either the + * old file or the new — never a torn read. + */ +export function saveCacConfig(projectDir, state, { log } = {}) { + const path = configPath(projectDir); + mkdirSync(dirname(path), { recursive: true }); + const coerced = coerceConfig(state); + const payload = JSON.stringify(coerced, null, 2); + const tmp = `${path}.tmp.${process.pid}.${Math.random().toString(36).slice(2)}`; + try { + writeFileSync(tmp, payload); + renameSync(tmp, path); + } catch (e) { + log?.(`cac-config: save failed (${e.message})`); + throw e; + } + return coerced; +} + +/** + * Patch one or more fields. Unknown fields are ignored (coerceConfig drops + * them). Returns the new state after persistence. + */ +export function updateCacConfig(projectDir, patch, { log } = {}) { + const current = loadCacConfig(projectDir, { log }); + const merged = { ...current, ...patch }; + return saveCacConfig(projectDir, merged, { log }); +} diff --git a/src/core/cac-predictor.js b/src/core/cac-predictor.js new file mode 100644 index 0000000..b90044a --- /dev/null +++ b/src/core/cac-predictor.js @@ -0,0 +1,573 @@ +/** + * CAC predictor — opt-in 4th gating axis for the diagnostic emit cascade. + * + * Given a diagnostic produced by the existing pipeline (severity → static + * confidence → adaptive-mode kill-switch → force-disable), this module + * predicts the probability that an agent will adopt the proposed fix and + * decides whether to: + * - allow the emit (prediction non-blocking), + * - downgrade its severity (de-emphasize without removing it), or + * - suppress it (drop entirely). + * + * The "neural" backend is a hierarchical empirical-Bayes scorer over the + * analytics store. It is INTENTIONALLY simple for the prototype: + * + * 1. Look up historical (rule_id, file_domain) outcomes — most specific. + * 2. Fall back to (rule_id) if (1) has fewer than `min_samples` outcomes. + * 3. Fall back to (severity) if (2) is also under-sampled. + * 4. If all three are under-sampled → allow (prediction has no signal). + * + * At each level we compute Beta(α, β) posteriors over `adopted / total` + * with a uniform prior (α = β = 2). The decision uses the posterior mean. + * The 95% credible interval is exposed for downstream telemetry / UI. + * + * The scorer is decoupled from the integration via a `historyProvider` + * dependency: the real provider queries the analytics store; tests inject a + * deterministic stub. This makes the gate logic unit-testable without a + * SQLite fixture. + * + * Safety contract — load-bearing: + * - Predictor only ever SUPPRESSES or DOWNGRADES; it never adds, mutates + * fix proposals, or alters params. If the gate is disabled, validate_code + * behavior is bit-identical to a build without this module. + * - When the gate raises (history provider crash, store unavailable), the + * diagnostic is allowed through unchanged. Failures degrade open. + * - In `shadow` mode, decisions are recorded for analysis but no + * diagnostic is mutated. Used to A/B-validate a threshold before + * flipping to `active`. + */ + +import { existsSync, readFileSync, readdirSync } from 'node:fs'; +import { join } from 'node:path'; +import { betaPosterior } from './analytics-queries.js'; +import { getDomainFromPath } from './domain-detector.js'; +import { readEvent } from './session-events.js'; + +const MAX_RECENT_DECISIONS = 200; +const PRIOR_A = 2; +const PRIOR_B = 2; + +const recentDecisions = []; + +/** + * The empirical-Bayes scorer. Pure function. + * + * @param {object} input + * @param {string} input.rule_id - Diagnostic rule_id (post-stamping). + * @param {string} input.severity - 'error' | 'warning' | 'info'. + * @param {string|null} input.file_domain - Output of getDomainFromPath, or null. + * @param {number} input.min_samples - Threshold below which a feature level is rejected. + * @param {(ruleId: string, fileDomain: string|null) => {adopted:number,total:number}} input.historyProvider + * @returns {{ + * p_adopted: number, + * p_lower: number, + * p_upper: number, + * n_samples: number, + * adopted: number, + * feature: 'rule_id+domain'|'rule_id'|'severity'|'prior', + * model: 'empirical_bayes_v1' + * }} + */ +export function scoreFixHelpfulness({ + rule_id, + severity, + file_domain, + min_samples, + historyProvider, + severityProvider, +}) { + const tries = []; + + if (rule_id && file_domain) { + const h = safeProvide(historyProvider, rule_id, file_domain); + tries.push({ feature: 'rule_id+domain', ...h }); + } + if (rule_id) { + const h = safeProvide(historyProvider, rule_id, null); + tries.push({ feature: 'rule_id', ...h }); + } + if (severity && severityProvider) { + const h = safeProvide(severityProvider, severity); + tries.push({ feature: 'severity', ...h }); + } + + // Pick the most specific feature with enough samples; the order in `tries` + // reflects the hierarchy. + const chosen = tries.find(t => t.total >= min_samples); + if (!chosen) { + // No level passed min_samples — no signal. Fall back to the prior, which + // for Beta(2, 2) is mean = 0.5. The decision layer treats `prior` as a + // pass-through (allow). + return { + p_adopted: 0.5, + p_lower: 0.0, + p_upper: 1.0, + n_samples: 0, + adopted: 0, + feature: 'prior', + model: 'empirical_bayes_v1', + }; + } + + const { mean, lower95, upper95 } = betaPosterior(chosen.adopted, chosen.total, PRIOR_A, PRIOR_B); + return { + p_adopted: mean, + p_lower: lower95, + p_upper: upper95, + n_samples: chosen.total, + adopted: chosen.adopted, + feature: chosen.feature, + model: 'empirical_bayes_v1', + }; +} + +function safeProvide(provider, ...args) { + try { + const out = provider(...args); + return { + adopted: Number.isFinite(out?.adopted) ? out.adopted : 0, + total: Number.isFinite(out?.total) ? out.total : 0, + }; + } catch { + return { adopted: 0, total: 0 }; + } +} + +/** + * Decide what to do with a diagnostic given a prediction and config. + * Pure function, separate from the prediction step so it can be unit-tested + * independently and tweaked without touching the scorer. + * + * Returns `{ decision, reason }` where decision is one of: + * - 'allow': emit unchanged + * - 'downgrade': emit but reduce severity (error→warning, warning→info) + * - 'suppress': drop emit entirely + */ +export function decideAction(prediction, config) { + // No signal → always allow. The predictor refuses to gate when it's flying + // blind — early in adoption, before enough outcomes accumulate, this is + // the safe default. + if (prediction.feature === 'prior') { + return { decision: 'allow', reason: 'no_signal' }; + } + if (prediction.p_adopted >= config.threshold) { + return { decision: 'allow', reason: 'above_threshold' }; + } + // Below threshold — apply the configured action. + return { + decision: config.action === 'suppress' ? 'suppress' : 'downgrade', + reason: 'below_threshold', + }; +} + +/** + * Real history provider over the analytics store. Returns + * `{ adopted, total }` for a rule_id, optionally segmented by file_domain. + * + * `adopted` = count of outcomes where fix_applied = 'verbatim'. + * `total` = count of outcomes for this rule_id (any fix_applied value). + * + * The file_domain filter uses LIKE patterns on `diagnostics.file` matching + * the same path heuristics as `domain-detector.js`. Only segments we can + * express cheaply in SQL are supported; unknown domains return `(0, 0)`. + */ +export function buildHistoryProvider(analyticsStore) { + if (!analyticsStore) { + return () => ({ adopted: 0, total: 0 }); + } + return function historyProvider(ruleId, fileDomain) { + if (!ruleId) return { adopted: 0, total: 0 }; + const pattern = fileDomain ? domainLikePattern(fileDomain) : null; + if (fileDomain && !pattern) return { adopted: 0, total: 0 }; + try { + const sql = pattern + ? `SELECT o.fix_applied, COUNT(*) as cnt + FROM outcomes o + WHERE EXISTS ( + SELECT 1 FROM diagnostics d + WHERE d.fp = o.fp + AND d.hint_rule_id = ? + AND d.suppressed = 0 + AND d.file LIKE ? + ) + GROUP BY o.fix_applied` + : `SELECT o.fix_applied, COUNT(*) as cnt + FROM outcomes o + WHERE EXISTS ( + SELECT 1 FROM diagnostics d + WHERE d.fp = o.fp + AND d.hint_rule_id = ? + AND d.suppressed = 0 + ) + GROUP BY o.fix_applied`; + const rows = pattern + ? analyticsStore.query(sql, [ruleId, pattern]) + : analyticsStore.query(sql, [ruleId]); + let total = 0; + let adopted = 0; + for (const row of rows) { + total += row.cnt; + if (row.fix_applied === 'verbatim') adopted += row.cnt; + } + return { adopted, total }; + } catch { + return { adopted: 0, total: 0 }; + } + }; +} + +/** + * Severity-level fallback provider. Less segmentation, more samples. + * Used when both rule_id+domain and rule_id alone are under-sampled. + */ +export function buildSeverityProvider(analyticsStore) { + if (!analyticsStore) { + return () => ({ adopted: 0, total: 0 }); + } + return function severityProvider(severity) { + try { + const rows = analyticsStore.query( + `SELECT o.fix_applied, COUNT(*) as cnt + FROM outcomes o + WHERE EXISTS ( + SELECT 1 FROM diagnostics d + WHERE d.fp = o.fp + AND d.severity = ? + AND d.suppressed = 0 + ) + GROUP BY o.fix_applied`, + [severity], + ); + let total = 0; + let adopted = 0; + for (const row of rows) { + total += row.cnt; + if (row.fix_applied === 'verbatim') adopted += row.cnt; + } + return { adopted, total }; + } catch { + return { adopted: 0, total: 0 }; + } + }; +} + +function domainLikePattern(domain) { + switch (domain) { + case 'commands': return '%/lib/commands/%'; + case 'queries': return '%/lib/queries/%'; + case 'pages': return '%/views/pages/%'; + case 'layouts': return '%/views/layouts/%'; + case 'partials': return '%/views/partials/%'; + case 'graphql': return '%/graphql/%'; + case 'schema': return '%/schema/%'; + case 'translations': return '%/translations/%'; + default: return null; + } +} + +const SEVERITY_RANK = ['info', 'warning', 'error']; + +function downgradeSeverity(s) { + const i = SEVERITY_RANK.indexOf(s); + if (i <= 0) return 'info'; + return SEVERITY_RANK[i - 1]; +} + +/** + * Apply the gate to a validate_code result. Mutates `result.errors`, + * `result.warnings`, `result.infos` in place when the gate is in `active` + * mode. In `shadow` mode the result is left untouched and decisions are + * appended to the session bus + recent-decisions ring buffer for later + * inspection. + * + * NEVER throws. If the predictor or store fails, the result passes through. + * + * @returns {Array} list of decisions emitted in this call (one per diagnostic + * considered). Order matches input order; useful for tests. + */ +export function applyCac(result, { + config, + analyticsStore, + filePath, + sessionBus, + log, + historyProvider, + severityProvider, +} = {}) { + if (!config?.enabled) return []; + if (!result) return []; + + const provider = historyProvider ?? buildHistoryProvider(analyticsStore); + const sevProvider = severityProvider ?? buildSeverityProvider(analyticsStore); + const fileDomain = filePath ? getDomainFromPath(filePath) : null; + const decisions = []; + + const buckets = [ + { name: 'errors', arr: result.errors ?? [] }, + { name: 'warnings', arr: result.warnings ?? [] }, + { name: 'infos', arr: result.infos ?? [] }, + ]; + + for (const bucket of buckets) { + const kept = []; + for (const d of bucket.arr) { + let decision; + try { + const rule_id = d.rule_id || (d.check ? `${d.check}.unmatched` : null); + const severity = d.severity || bucketToSeverity(bucket.name); + const prediction = scoreFixHelpfulness({ + rule_id, + severity, + file_domain: fileDomain, + min_samples: config.min_samples, + historyProvider: provider, + severityProvider: sevProvider, + }); + decision = decideAction(prediction, config); + recordDecision({ + file: filePath, + rule_id, + check: d.check, + severity, + file_domain: fileDomain, + prediction, + decision, + mode: config.mode, + }, sessionBus); + decisions.push({ rule_id, check: d.check, prediction, decision }); + } catch (e) { + log?.(`cac-predictor: scoring failed (${e?.message ?? e}); allowing diagnostic`); + kept.push(d); + continue; + } + + // Shadow mode: never modifies the result. + if (config.mode !== 'active') { + kept.push(d); + continue; + } + + if (decision.decision === 'suppress') { + // drop — do not push + continue; + } + if (decision.decision === 'downgrade') { + const next = downgradeSeverity(d.severity || bucketToSeverity(bucket.name)); + if (next !== d.severity) { + d.severity = next; + d.cac_downgraded = true; + } + kept.push(d); + continue; + } + kept.push(d); + } + bucket.arr.length = 0; + bucket.arr.push(...kept); + } + + // Active-mode downgrades may have flipped severities; rebalance the + // buckets so an error→warning downgrade actually moves into result.warnings + // and not just gets stamped with severity:'warning' in result.errors. + if (config.mode === 'active') { + rebalanceBuckets(result); + } + + return decisions; +} + +function bucketToSeverity(name) { + if (name === 'errors') return 'error'; + if (name === 'warnings') return 'warning'; + return 'info'; +} + +function rebalanceBuckets(result) { + const all = [ + ...((result.errors ?? []).map(d => ({ d, defaultBucket: 'errors' }))), + ...((result.warnings ?? []).map(d => ({ d, defaultBucket: 'warnings' }))), + ...((result.infos ?? []).map(d => ({ d, defaultBucket: 'infos' }))), + ]; + result.errors = []; + result.warnings = []; + result.infos = []; + for (const { d, defaultBucket } of all) { + const sev = d.severity || bucketToSeverity(defaultBucket); + if (sev === 'error') result.errors.push(d); + else if (sev === 'warning') result.warnings.push(d); + else result.infos.push(d); + } +} + +function recordDecision(entry, sessionBus) { + // The session-bus envelope owns `ts` (it's a reserved envelope key — see + // `session-events.js::ENVELOPE_KEYS`). Compute it once, pass it as the + // emit's third arg, and keep a copy on the ring entry so consumers of + // `getRecentCacDecisions()` see a single self-contained record without + // re-querying the bus. + const ts = new Date().toISOString(); + const payload = { + file: entry.file ?? null, + rule_id: entry.rule_id ?? null, + check: entry.check ?? null, + severity: entry.severity, + file_domain: entry.file_domain ?? null, + p_adopted: entry.prediction?.p_adopted ?? null, + p_lower: entry.prediction?.p_lower ?? null, + p_upper: entry.prediction?.p_upper ?? null, + n_samples: entry.prediction?.n_samples ?? 0, + feature: entry.prediction?.feature ?? 'prior', + decision: entry.decision?.decision ?? 'allow', + reason: entry.decision?.reason ?? '', + mode: entry.mode, + }; + pushRingEntry({ ts, ...payload }); + if (sessionBus?.emit) { + try { + sessionBus.emit('cac_decision', payload, ts); + } catch { + // Persistence is best-effort. The in-memory ring already received the + // entry, so the dashboard still shows it within this session. + } + } +} + +function pushRingEntry(entry) { + recentDecisions.push(entry); + if (recentDecisions.length > MAX_RECENT_DECISIONS) { + recentDecisions.splice(0, recentDecisions.length - MAX_RECENT_DECISIONS); + } +} + +export function getRecentCacDecisions(limit = MAX_RECENT_DECISIONS) { + const start = Math.max(0, recentDecisions.length - limit); + return recentDecisions.slice(start); +} + +export function clearRecentCacDecisions() { + recentDecisions.length = 0; +} + +/** + * Reconstruct an in-memory ring entry from a persisted `cac_decision` + * envelope. The persisted shape places the timestamp on the envelope + * (`event.ts`) and every other field as a top-level payload key (validated + * by the registry). The ring entry collapses both back into a single flat + * object so consumers of `getRecentCacDecisions()` see a uniform record + * regardless of whether it came from the live emit path or from disk. + */ +function eventToRingEntry(event) { + return { + ts: event.ts, + file: event.file ?? null, + rule_id: event.rule_id ?? null, + check: event.check ?? null, + severity: event.severity, + file_domain: event.file_domain ?? null, + p_adopted: event.p_adopted, + p_lower: event.p_lower, + p_upper: event.p_upper, + n_samples: event.n_samples, + feature: event.feature, + decision: event.decision, + reason: event.reason, + mode: event.mode, + }; +} + +/** + * Scan one NDJSON file for `cac_decision` events and return ring-shape + * entries. Tolerates malformed lines (skipped silently — we don't want a + * single corrupt event to nuke the rehydration of an otherwise valid log). + * + * Performance shortcut: most lines won't be `cac_decision`, so peek at the + * `kind` field via cheap JSON.parse before paying the full Zod validation + * cost in `readEvent`. + */ +function extractCacDecisionsFromFile(filePath) { + if (!existsSync(filePath)) return []; + let content; + try { content = readFileSync(filePath, 'utf-8'); } + catch { return []; } + const out = []; + const lines = content.split('\n'); + for (const line of lines) { + if (!line || !line.trim()) continue; + let raw; + try { raw = JSON.parse(line); } + catch { continue; } + if (!raw || raw.kind !== 'cac_decision') continue; + try { + const event = readEvent(line); + out.push(eventToRingEntry(event)); + } catch { + // Malformed payload (e.g. older shape from a pre-schema version of + // this code). Skip — the dashboard would rather see fewer entries + // than crash the predictor on a corrupt line. + } + } + return out; +} + +/** + * Read recent session NDJSON logs and return the last `limit` + * `cac_decision` entries in chronological order (oldest → newest). + * + * Scans `//events.ndjson`, sorted by directory + * name DESC (session ids are ISO timestamps so lexical order = chronological). + * Stops as soon as we have ≥ `2 × limit` candidates collected — overscan + * guards against the same event appearing twice across boundary cases + * (e.g. an in-flight session being scanned both by us and the live writer) + * while still bounding the I/O at the most recent few sessions. + * + * Returns [] for any non-fatal failure (missing dir, unreadable, etc.). + * Never throws — server startup must not be blocked by a broken sessions + * directory. + * + * @param {string} sessionsDir + * @param {number} [limit=MAX_RECENT_DECISIONS] + * @returns {Array} + */ +export function loadRecentCacDecisions(sessionsDir, limit = MAX_RECENT_DECISIONS) { + if (!sessionsDir || limit <= 0) return []; + let entries; + try { entries = readdirSync(sessionsDir, { withFileTypes: true }); } + catch { return []; } + + const sessionDirs = entries + .filter(e => e.isDirectory()) + .map(e => e.name) + .sort() + .reverse(); // newest session first (lexical = chronological) + + const collected = []; + const overscanCap = limit * 2; + for (const name of sessionDirs) { + if (collected.length >= overscanCap) break; + const filePath = join(sessionsDir, name, 'events.ndjson'); + const fromFile = extractCacDecisionsFromFile(filePath); + for (const e of fromFile) collected.push(e); + } + + // Final ordering and trim. The bus envelope stamps `ts` as an ISO string; + // lexical compare matches chronological order. Stable: same-ts entries + // keep their original (file scan) order. + collected.sort((a, b) => (a.ts < b.ts ? -1 : a.ts > b.ts ? 1 : 0)); + return collected.slice(Math.max(0, collected.length - limit)); +} + +/** + * Replace the in-memory ring with the most recent `cac_decision` entries + * persisted on disk. Returns the number of entries loaded so callers can + * log a one-line "rehydrated N from disk" startup message. + * + * Idempotent — calling twice produces the same end state. Safe to call + * before any live emits (server boot) but not while emits are in flight, + * since this overwrites the ring rather than merging. + */ +export function rehydrateRecentCacDecisions(sessionsDir, limit = MAX_RECENT_DECISIONS) { + const loaded = loadRecentCacDecisions(sessionsDir, limit); + recentDecisions.length = 0; + for (const e of loaded) recentDecisions.push(e); + return loaded.length; +} diff --git a/src/core/case-base.js b/src/core/case-base.js new file mode 100644 index 0000000..55f2daf --- /dev/null +++ b/src/core/case-base.js @@ -0,0 +1,684 @@ +/** + * Case base — closed-loop learning from diagnostic outcomes. + * + * F1: Case retrieval — given a (check, template_fp), what fixes have been + * applied and what were their outcomes? Returns ranked candidates. + * + * F2: Rule scoring — per-rule rolling stats (emitted, adopted, resolved, + * regressed). Rules can query this to adjust confidence. Rules below + * threshold are flagged for disabling. + * + * F3: Suggested-rule synthesis — identify diagnostics with no matching rule + * but a clear case-base signal. Output: structured suggestion for human + * review (never auto-applied). + * + * All functions take an opened analytics store. Case base is a read-only + * view over the existing tables — no additional schema needed. + */ + +import { classifyFileType } from './rules/queries.js'; + +const MIN_CASES = 3; +const RULE_DISABLE_THRESHOLD = 0.15; +const GUARD_MIN_SAMPLES = 5; + +/** + * Resolve the tri-state `since` parameter to an ISO string or null. + * See analytics-queries.js:resolveSince — same contract, kept private to + * each module so the case-base never imports from analytics-queries (the + * dependency would invert the layering: case-base feeds engine state + * decisions; analytics-queries feeds reporting). + * + * Reporting callers in case-base (retrieveCases*, suggestedRules, + * synthesizeGuardPredicate, ruleScores via /api/analytics/rule-scores) + * pass `since` and let it auto-resolve to the operator baseline. + * + * Engine-state callers (`syncDisabledRules` in server.js, + * `disabled_rules` in server-status.js, `resolveProbation` here) pass + * `since: null` explicitly so a baseline operator-set on the dashboard + * never narrows the data the auto-disable / probation logic sees. + * + * `scoreRule` is the live confidence-adjustment path (called per emit + * from rules/engine.js) — it deliberately has no `since` parameter at + * all; case-base scoring must always see full history. + */ +function resolveSince(store, since) { + if (since === null) return null; + if (typeof since === 'string' && since.length > 0) return since; + if (store && typeof store.getBaselineTs === 'function') { + try { + const baseline = store.getBaselineTs(); + return baseline ?? null; + } catch { + return null; + } + } + return null; +} + +/** + * F1: Retrieve cases for a diagnostic template. + * + * "For diagnostics matching (check, template_fp), what fixes were applied + * and how did they turn out?" + * + * @param {object} store - Opened analytics store + * @param {string} check - Check name (e.g. 'UnknownFilter') + * @param {string} templateFp - Template fingerprint + * @param {object} [opts] + * @param {number} [opts.minCases=3] - Minimum cases to include a fix + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {CaseResult} + */ +export function retrieveCases(store, check, templateFp, { minCases = MIN_CASES, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND ts >= ?' : ''; + const sinceAndD = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const totalRow = store.queryOne(` + SELECT COUNT(*) as cnt FROM diagnostics + WHERE check_name = ? AND template_fp = ? AND suppressed = 0 ${sinceAnd} + `, [check, templateFp, ...sinceP]); + + const total = totalRow?.cnt ?? 0; + if (total === 0) return { check, template_fp: templateFp, total: 0, cases: [] }; + + const outcomeRows = store.query(` + SELECT o.outcome, o.fix_applied, o.collateral_added, COUNT(*) as cnt + FROM outcomes o + JOIN diagnostics d ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.check_name = ? AND d.template_fp = ? ${sinceAndD} + GROUP BY o.outcome, o.fix_applied + ORDER BY cnt DESC + `, [check, templateFp, ...sinceP]); + + const byFix = new Map(); + for (const row of outcomeRows) { + const key = row.fix_applied ?? '__none__'; + if (!byFix.has(key)) { + byFix.set(key, { fix_applied: row.fix_applied, resolved: 0, regressed: 0, unchanged: 0, moved: 0, total: 0, collateral: 0 }); + } + const entry = byFix.get(key); + entry[row.outcome] = (entry[row.outcome] ?? 0) + row.cnt; + entry.total += row.cnt; + if (row.outcome === 'regressed') entry.collateral += row.collateral_added * row.cnt; + } + + const cases = [...byFix.values()] + .filter(c => c.total >= minCases) + .map(c => ({ + fix_applied: c.fix_applied, + total: c.total, + resolved: c.resolved, + regressed: c.regressed, + resolution_rate: c.total > 0 ? c.resolved / c.total : 0, + regression_rate: c.total > 0 ? c.regressed / c.total : 0, + avg_collateral: c.regressed > 0 ? c.collateral / c.regressed : 0, + })) + .sort((a, b) => b.resolution_rate - a.resolution_rate || a.regression_rate - b.regression_rate); + + return { check, template_fp: templateFp, total, cases }; +} + +/** + * F1 (batch): Retrieve top cases for all templates of a given check. + * + * @param {object} [opts] + * @param {number} [opts.minCases=3] + * @param {number} [opts.limit=20] + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + */ +export function retrieveCasesByCheck(store, check, { minCases = MIN_CASES, limit = 20, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const templates = store.query(` + SELECT DISTINCT template_fp FROM diagnostics + WHERE check_name = ? AND template_fp IS NOT NULL AND suppressed = 0 ${sinceAnd} + `, [check, ...sinceP]); + + const results = []; + for (const { template_fp } of templates) { + // Forward the *resolved* timestamp (or null) so the inner call doesn't + // re-read meta. Without this, an explicit since='ISO' here would still + // propagate as undefined inside retrieveCases and pull from meta — which + // is the wrong source when the operator is overriding the default. + const caseResult = retrieveCases(store, check, template_fp, { minCases, since: sinceTs }); + if (caseResult.cases.length > 0) results.push(caseResult); + } + + results.sort((a, b) => b.total - a.total); + return results.slice(0, limit); +} + +/** + * F2: Per-rule rolling stats — **promotion-gating view.** + * + * Drives `syncDisabledRules` and probation resolution. Default threshold is 5 + * because promotion decisions must be statistically meaningful — one bad + * resolution on a rule that has only fired twice is not evidence to disable it. + * Reporting views that want to list every rule regardless of sample size use + * `rulePerformance()` in analytics-queries.js (threshold 1, no `disabled` flag). + * + * `${check}.unmatched` fallback rule_ids (set by the diagnostic pipeline for + * rule-less diagnostics — A4) are excluded here: they don't correspond to a + * registered rule, so "disabling" them has no effect and they shouldn't count + * toward promotion/probation decisions. + * + * `since` contract — IMPORTANT for engine-state callers: + * - server.js `syncDisabledRules` and tools/server-status.js's + * `disabled_rules` snapshot pass `since: null` explicitly so the + * operator-set reporting baseline cannot narrow what auto-disable + * considers. A sample of "since baseline" might be too small to + * trigger correctly; full history is required for stable engine state. + * - http-server.js endpoints (rule-scores, engine-map) omit `since` + * so they default to the resolved meta baseline — operators viewing + * the dashboard see the post-baseline view. + * + * @param {object} store + * @param {object} [opts] + * @param {number} [opts.minEmitted=5] - Minimum emissions to include + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * ENGINE-STATE callers must pass `null` to bypass the meta baseline. + * @returns {Array} + */ +export function ruleScores(store, { minEmitted = 5, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceAndD = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const ruleRows = store.query(` + SELECT d.hint_rule_id as rule_id, + COUNT(*) as emitted, + d.check_name as check_name + FROM diagnostics d + WHERE d.hint_rule_id IS NOT NULL + AND d.hint_rule_id != 'unknown' + AND d.hint_rule_id NOT LIKE '%.unmatched' + AND d.suppressed = 0 + ${sinceAnd} + GROUP BY d.hint_rule_id + HAVING COUNT(*) >= ? + `, [...sinceP, minEmitted]); + + const scores = []; + + for (const row of ruleRows) { + const outcomeRows = store.query(` + SELECT o.outcome, o.fix_applied, COUNT(*) as cnt + FROM outcomes o + WHERE o.fp IN ( + SELECT DISTINCT d.fp FROM diagnostics d + WHERE d.hint_rule_id = ? AND d.suppressed = 0 ${sinceAndD} + ) + GROUP BY o.outcome, o.fix_applied + `, [row.rule_id, ...sinceP]); + + let resolved = 0, regressed = 0, unchanged = 0, moved = 0; + let adopted = 0, totalOutcomes = 0; + + for (const o of outcomeRows) { + totalOutcomes += o.cnt; + if (o.outcome === 'resolved') resolved += o.cnt; + else if (o.outcome === 'regressed') regressed += o.cnt; + else if (o.outcome === 'unchanged') unchanged += o.cnt; + else if (o.outcome === 'moved') moved += o.cnt; + if (o.fix_applied === 'verbatim') adopted += o.cnt; + } + + const resolutionRate = totalOutcomes > 0 ? resolved / totalOutcomes : 0; + const regressionRate = totalOutcomes > 0 ? regressed / totalOutcomes : 0; + const adoptionRate = totalOutcomes > 0 ? adopted / totalOutcomes : 0; + const effectivenessScore = resolutionRate - regressionRate; + + scores.push({ + rule_id: row.rule_id, + check: row.check_name, + emitted: row.emitted, + total_outcomes: totalOutcomes, + resolved, + regressed, + unchanged, + moved, + adopted, + resolution_rate: resolutionRate, + regression_rate: regressionRate, + adoption_rate: adoptionRate, + effectiveness: effectivenessScore, + disabled: effectivenessScore < RULE_DISABLE_THRESHOLD && totalOutcomes >= 10, + }); + } + + scores.sort((a, b) => a.effectiveness - b.effectiveness); + return scores; +} + +/** + * F2: Compute confidence adjustment for a specific rule given a template_fp. + * + * Rules call this during `apply()` to boost or reduce their confidence + * based on historical outcomes. + * + * @param {object} store + * @param {string} ruleId + * @param {string} templateFp + * @returns {{ adjustment: number, reason: string } | null} + */ +export function scoreRule(store, ruleId, templateFp) { + const stats = store.queryOne(` + SELECT COUNT(DISTINCT d.fp) as emitted + FROM diagnostics d + WHERE d.hint_rule_id = ? AND d.template_fp = ? AND d.suppressed = 0 + `, [ruleId, templateFp]); + + if (!stats || stats.emitted < MIN_CASES) return null; + + const outcomes = store.query(` + SELECT o.outcome, COUNT(*) as cnt + FROM outcomes o + JOIN diagnostics d ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.hint_rule_id = ? AND d.template_fp = ? + GROUP BY o.outcome + `, [ruleId, templateFp]); + + let resolved = 0, regressed = 0, total = 0; + for (const o of outcomes) { + total += o.cnt; + if (o.outcome === 'resolved') resolved += o.cnt; + if (o.outcome === 'regressed') regressed += o.cnt; + } + + if (total === 0) return null; + + const resRate = resolved / total; + const regRate = regressed / total; + + if (resRate >= 0.7) { + return { adjustment: +0.1, reason: `${(resRate * 100).toFixed(0)}% resolution rate for this template` }; + } + if (regRate >= 0.3) { + return { adjustment: -0.2, reason: `${(regRate * 100).toFixed(0)}% regression rate — hint may be harmful` }; + } + return { adjustment: 0, reason: 'insufficient signal' }; +} + +/** + * F3: Identify diagnostics with no matching rule but a clear case-base + * signal (high resolution rate on some fix pattern). These are candidates + * for new rules. + * + * @param {object} store + * @param {Set} [existingRuleChecks] - Checks that already have rules + * @param {object} [opts] + * @param {number} [opts.minCases=5] - Minimum outcome count + * @param {number} [opts.minResolutionRate=0.6] - Minimum resolution rate + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {Array} + */ +export function suggestedRules(store, existingRuleChecks = new Set(), { minCases = 5, minResolutionRate = 0.6, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND d.ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const candidates = store.query(` + SELECT d.check_name, d.template_fp, + COUNT(DISTINCT d.fp) as emitted, + GROUP_CONCAT(DISTINCT d.hint_rule_id) as rule_ids + FROM diagnostics d + WHERE d.suppressed = 0 AND d.template_fp IS NOT NULL ${sinceAnd} + GROUP BY d.check_name, d.template_fp + HAVING COUNT(DISTINCT d.fp) >= ? + `, [...sinceP, minCases]); + + const suggestions = []; + + for (const c of candidates) { + const hasRule = c.rule_ids && c.rule_ids !== 'unknown' && c.rule_ids.split(',').some(r => r !== 'unknown'); + if (hasRule) continue; + + const outcomeRows = store.query(` + SELECT o.outcome, o.fix_applied, COUNT(*) as cnt + FROM outcomes o + JOIN diagnostics d ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.check_name = ? AND d.template_fp = ? ${sinceAnd} + GROUP BY o.outcome, o.fix_applied + `, [c.check_name, c.template_fp, ...sinceP]); + + let resolved = 0, total = 0; + let bestFix = null; + let bestFixResolved = 0; + + for (const o of outcomeRows) { + total += o.cnt; + if (o.outcome === 'resolved') { + resolved += o.cnt; + if (o.fix_applied && o.cnt > bestFixResolved) { + bestFix = o.fix_applied; + bestFixResolved = o.cnt; + } + } + } + + if (total < minCases) continue; + const resRate = resolved / total; + if (resRate < minResolutionRate) continue; + + // The sample diagnostic must come from the same window the suggestion + // was derived from; if the operator is viewing post-baseline only, the + // sample file should reflect that window too. + const sampleSinceAnd = sinceTs ? 'AND ts >= ?' : ''; + const sampleDiag = store.queryOne(` + SELECT file, check_name, fp FROM diagnostics + WHERE check_name = ? AND template_fp = ? AND suppressed = 0 ${sampleSinceAnd} + ORDER BY ts DESC LIMIT 1 + `, [c.check_name, c.template_fp, ...sinceP]); + + suggestions.push({ + check: c.check_name, + template_fp: c.template_fp, + emitted: c.emitted, + total_outcomes: total, + resolved, + resolution_rate: resRate, + best_fix: bestFix, + best_fix_count: bestFixResolved, + sample_file: sampleDiag?.file ?? null, + suggestion: `Check \`${c.check_name}\` (template ${c.template_fp.slice(0, 8)}) has ${(resRate * 100).toFixed(0)}% resolution rate across ${total} outcomes but no rule. Consider adding a rule in \`src/core/rules/${c.check_name}.js\`.`, + }); + } + + suggestions.sort((a, b) => b.resolution_rate - a.resolution_rate || b.emitted - a.emitted); + return suggestions; +} + +/** + * F4: Synthesize guard predicates from historical diagnostic data. + * + * Analyzes patterns in file paths and diagnostic params to produce a `when` + * object compatible with promoted-rules JSON format (see compileWhen()). + * + * Thresholds: + * param_equals — ≥90% of values identical + * param_startsWith — ≥80% share a common prefix (len ≥ 2) + * param_contains — ≥80% contain a common substring (len ≥ 3) + * file_type — ≥80% share the same classified type + * + * @param {object} store - Analytics store + * @param {string} check - Check name + * @param {string} templateFp - Template fingerprint + * @param {object} [opts] + * @param {number} [opts.minSamples=5] - Minimum samples to infer a guard + * @param {string|null} [opts.since] - Reporting baseline. See resolveSince(). + * @returns {object} JSON `when` object for promoted rules + */ +export function synthesizeGuardPredicate(store, check, templateFp, { minSamples = GUARD_MIN_SAMPLES, since } = {}) { + const sinceTs = resolveSince(store, since); + const sinceAnd = sinceTs ? 'AND ts >= ?' : ''; + const sinceP = sinceTs ? [sinceTs] : []; + + const when = {}; + + const fileRows = store.query(` + SELECT DISTINCT file FROM diagnostics + WHERE check_name = ? AND template_fp = ? AND suppressed = 0 ${sinceAnd} + `, [check, templateFp, ...sinceP]); + + if (fileRows.length >= minSamples) { + const types = fileRows.map(r => classifyFileType(r.file)); + const dominant = dominantValue(types); + if (dominant && dominant.ratio >= 0.8 && dominant.value !== 'unknown') { + when.file_type = dominant.value; + } + } + + const eventRows = store.query(` + SELECT payload FROM events + WHERE kind = 'validator_emit' + AND json_extract(payload, '$.check') = ? + AND json_extract(payload, '$.template_fp') = ? + ${sinceAnd} + `, [check, templateFp, ...sinceP]); + + const paramSamples = []; + for (const row of eventRows) { + try { + const p = JSON.parse(row.payload); + if (p.params && typeof p.params === 'object' && Object.keys(p.params).length > 0) { + paramSamples.push(p.params); + } + } catch { /* skip malformed */ } + } + + if (paramSamples.length >= minSamples) { + const keys = new Set(); + for (const s of paramSamples) { + for (const k of Object.keys(s)) keys.add(k); + } + + for (const key of keys) { + const values = paramSamples + .map(s => s[key]) + .filter(v => typeof v === 'string' && v.length > 0); + + if (values.length < minSamples) continue; + + const top = dominantValue(values); + if (top && top.ratio >= 0.9) { + if (!when.param_equals) when.param_equals = {}; + when.param_equals[key] = top.value; + continue; + } + + const prefix = findDominantPrefix(values, 2); + if (prefix) { + if (!when.param_startsWith) when.param_startsWith = {}; + when.param_startsWith[key] = prefix; + continue; + } + + const substr = findDominantSubstring(values, 3); + if (substr) { + if (!when.param_contains) when.param_contains = {}; + when.param_contains[key] = substr; + } + } + } + + return when; +} + +/** + * F3: Generate a rule template for a suggested rule. + * Produces a JS code template that can be saved as a draft for human review. + * + * @param {object} suggestion - From suggestedRules() + * @param {object} [guards={}] - Synthesized when clause from synthesizeGuardPredicate() + */ +export function generateRuleTemplate(suggestion, guards = {}) { + const { check, template_fp } = suggestion; + const safeCheck = check.replace(/[^a-zA-Z0-9_]/g, '_'); + const shortFp = template_fp.slice(0, 8); + + const whenBody = Object.keys(guards).length > 0 + ? renderGuardsAsJs(guards) + : ' // TODO: Add guard predicate based on diagnostic params\n return true;'; + + return `// Suggested rule — generated from case-base analysis +// Template fingerprint: ${template_fp} +// Resolution rate: ${(suggestion.resolution_rate * 100).toFixed(0)}% across ${suggestion.total_outcomes} outcomes +// Sample file: ${suggestion.sample_file ?? 'unknown'} +// +// Review this rule before merging. Never auto-merge suggested rules. + +{ + id: '${safeCheck}.case_${shortFp}', + check: '${check}', + priority: 50, + when: (diag) => { +${whenBody} + }, + apply: (diag) => { + return { + rule_id: '${safeCheck}.case_${shortFp}', + hint_md: 'TODO: Write hint based on case-base patterns', + fixes: [], + confidence: ${suggestion.resolution_rate.toFixed(2)}, + }; + }, +}`; +} + +/** + * J5: Resolve probation for promoted rules. + * + * For each rule on probation, check if it has accumulated enough outcomes + * to make a determination. Compares the promoted rule's effectiveness to + * the disable threshold. + * + * @param {object} store - Analytics store + * @param {object} [opts] + * @param {number} [opts.minOutcomes=20] - Minimum outcomes before resolving + * @returns {Array<{ rule_id, resolution, effectiveness, outcomes }>} + */ +export function resolveProbation(store, { minOutcomes = 20 } = {}) { + const onProbation = store.getPromotionsOnProbation(); + const resolutions = []; + + for (const promo of onProbation) { + const outcomeRows = store.query(` + SELECT o.outcome, COUNT(*) as cnt + FROM outcomes o + JOIN diagnostics d ON o.fp = d.fp AND o.session_id = d.session_id AND o.file = d.file + WHERE d.hint_rule_id = ? + GROUP BY o.outcome + `, [promo.rule_id]); + + let resolved = 0, regressed = 0, total = 0; + for (const o of outcomeRows) { + total += o.cnt; + if (o.outcome === 'resolved') resolved += o.cnt; + if (o.outcome === 'regressed') regressed += o.cnt; + } + + if (total < minOutcomes) continue; + + const effectiveness = total > 0 ? (resolved - regressed) / total : 0; + let resolution; + + if (effectiveness < RULE_DISABLE_THRESHOLD) { + resolution = 'disabled'; + } else { + resolution = 'kept'; + } + + store.resolvePromotion(promo.rule_id, resolution); + resolutions.push({ + rule_id: promo.rule_id, + resolution, + effectiveness, + outcomes: total, + }); + } + + return resolutions; +} + +// ── Guard synthesis helpers ──────────────────────────────────────────────── + +function dominantValue(arr) { + const freq = new Map(); + for (const v of arr) freq.set(v, (freq.get(v) || 0) + 1); + let best = null; + for (const [value, count] of freq) { + if (!best || count > best.count) best = { value, count }; + } + return best ? { value: best.value, count: best.count, ratio: best.count / arr.length } : null; +} + +function findDominantPrefix(values, minLen) { + if (values.length === 0) return null; + const threshold = values.length * 0.8; + let best = null; + for (const val of values) { + for (let len = val.length; len >= minLen; len--) { + const prefix = val.slice(0, len); + if (best && prefix.length <= best.length) break; + const matchCount = values.filter(v => v.startsWith(prefix)).length; + if (matchCount >= threshold) { + best = prefix; + break; + } + } + } + return best; +} + +function findDominantSubstring(values, minLen) { + if (values.length === 0) return null; + const shortest = values.reduce((a, b) => a.length <= b.length ? a : b); + let best = null; + for (let len = shortest.length; len >= minLen; len--) { + for (let start = 0; start <= shortest.length - len; start++) { + const candidate = shortest.slice(start, start + len); + const matchCount = values.filter(v => v.includes(candidate)).length; + if (matchCount / values.length >= 0.8) { + if (!best || candidate.length > best.length) best = candidate; + return best; + } + } + } + return best; +} + +const FILE_TYPE_PATH_HINT = { + page: '/pages/', + partial: '/partials/', + layout: '/layouts/', + command: '/commands/', + query: '/queries/', + graphql: '/graphql/', + schema: '/schema/', + module: 'modules/', +}; + +function renderGuardsAsJs(guards) { + const conditions = []; + + if (guards.param_equals) { + for (const [k, v] of Object.entries(guards.param_equals)) { + conditions.push(`diag.params?.${k} === ${JSON.stringify(v)}`); + } + } + + if (guards.param_startsWith) { + for (const [k, v] of Object.entries(guards.param_startsWith)) { + conditions.push(`diag.params?.${k}?.startsWith(${JSON.stringify(v)})`); + } + } + + if (guards.param_contains) { + for (const [k, v] of Object.entries(guards.param_contains)) { + conditions.push(`diag.params?.${k}?.includes(${JSON.stringify(v)})`); + } + } + + if (guards.file_type) { + const hint = FILE_TYPE_PATH_HINT[guards.file_type]; + if (hint) { + conditions.push(`diag.file?.includes(${JSON.stringify(hint)})`); + } + } + + if (conditions.length === 0) { + return ' // No guards synthesized — review and narrow this rule\n return true;'; + } + + if (conditions.length === 1) { + return ` return ${conditions[0]};`; + } + + return ` return ${conditions.join(' &&\n ')};`; +} diff --git a/src/core/constants.js b/src/core/constants.js index ea8419f..0531a3d 100644 --- a/src/core/constants.js +++ b/src/core/constants.js @@ -36,6 +36,33 @@ export const FILTER_MATCH_MAX_DISTANCE = 2; /** After this many consecutive non-decreasing error counts, warn the agent. */ export const CONSECUTIVE_ERROR_THRESHOLD = 3; +// ── Confidence defaults ───────────────────────────────────────────────────── + +/** + * Default confidence for a diagnostic when the rule engine did not set one. + * + * These are coarse priors: errors are high-confidence (the linter is usually + * right about real bugs), warnings are mid-confidence (more stylistic / context + * dependent), infos are low-confidence (advisory). + * + * A populated confidence — even a default — lets confidenceCalibration bucket + * every diagnostic instead of silently dropping ones where no rule matched. + * Case-base scoring can still override for rule-matched diagnostics. + */ +export const DEFAULT_CONFIDENCE_BY_SEVERITY = { + error: 0.9, + warning: 0.7, + info: 0.5, +}; + +/** + * Default confidence for pos-supervisor structural warnings (check names + * prefixed with `pos-supervisor:`). These are AST-derived, not LSP-derived, + * and are more deterministic than severity alone suggests — they only fire + * when the structural rule is actually hit. + */ +export const STRUCTURAL_DEFAULT_CONFIDENCE = 0.75; + // ── Limits ────────────────────────────────────────────────────────────────── /** Max subprocess output buffer (pos-cli check). */ diff --git a/src/core/dependency-graph.js b/src/core/dependency-graph.js index d91a88d..5384cb4 100644 --- a/src/core/dependency-graph.js +++ b/src/core/dependency-graph.js @@ -64,7 +64,7 @@ export function resolveRenderTarget(name, projectMap, callerPath) { if (partial?.path) return partial.path; // Fallback: assume the standard location. May produce a non-existent path - // when the render name is wrong — dead-code detection treats that as an + // when the render name is wrong — orphan detection treats that as an // edge into a missing file, which surfaces as broken_render from the // existing integrity checks. return `app/views/partials/${resolved}.liquid`; @@ -133,6 +133,7 @@ export function buildDependencyGraph(projectMap, lspResults = {}) { }; for (const page of Object.values(projectMap.pages ?? {})) seed(page.path); for (const partial of Object.values(projectMap.partials ?? {})) seed(partial.path); + for (const layout of Object.values(projectMap.layouts ?? {})) seed(layout.path); for (const cmd of Object.keys(projectMap.commands ?? {})) seed(cmd); for (const q of Object.keys(projectMap.queries ?? {})) seed(q); for (const g of Object.keys(projectMap.graphql ?? {})) seed(`app/graphql/${g}.graphql`); @@ -145,7 +146,17 @@ export function buildDependencyGraph(projectMap, lspResults = {}) { if (!graph[target].referenced_by.includes(source)) graph[target].referenced_by.push(source); }; - // ── 1. Walk pages ── + // ── 1. Walk pages (render/function edges + layout reference) ── + const layoutsByName = {}; + for (const layout of Object.values(projectMap.layouts ?? {})) { + if (!layout?.path) continue; + const name = layout.path + .replace(/^app\/views\/layouts\//, '') + .replace(/\.html\.liquid$/, '') + .replace(/\.liquid$/, ''); + layoutsByName[name] = layout.path; + } + for (const page of Object.values(projectMap.pages ?? {})) { if (!page?.path) continue; for (const r of page.renders ?? []) { @@ -154,9 +165,24 @@ export function buildDependencyGraph(projectMap, lspResults = {}) { for (const fc of page.function_calls ?? []) { addEdge(page.path, resolveFunctionTarget(fc.path)); } + if (page.layout) { + const layoutPath = layoutsByName[page.layout]; + if (layoutPath) addEdge(page.path, layoutPath); + } + } + + // ── 2. Walk layouts (render/function edges) ── + for (const layout of Object.values(projectMap.layouts ?? {})) { + if (!layout?.path) continue; + for (const r of layout.renders ?? []) { + addEdge(layout.path, resolveRenderTarget(r, projectMap, layout.path)); + } + for (const fc of layout.function_calls ?? []) { + addEdge(layout.path, resolveFunctionTarget(fc.path)); + } } - // ── 2. Walk partials ── + // ── 3. Walk partials ── for (const partial of Object.values(projectMap.partials ?? {})) { if (!partial?.path) continue; for (const r of partial.renders ?? []) { @@ -167,7 +193,7 @@ export function buildDependencyGraph(projectMap, lspResults = {}) { } } - // ── 3. Walk commands ── + // ── 4. Walk commands ── // Commands can call sub-commands (build/check phases) via {% function %} // and call GraphQL ops via {% graphql %}. for (const [cmdPath, cmd] of Object.entries(projectMap.commands ?? {})) { @@ -180,7 +206,7 @@ export function buildDependencyGraph(projectMap, lspResults = {}) { } } - // ── 4. Walk queries ── + // ── 5. Walk queries ── for (const [qPath, q] of Object.entries(projectMap.queries ?? {})) { for (const fc of q.function_calls ?? []) { addEdge(qPath, resolveFunctionTarget(fc.path)); @@ -191,7 +217,7 @@ export function buildDependencyGraph(projectMap, lspResults = {}) { } } - // ── 5. Merge LSP-derived edges on top ── + // ── 6. Merge LSP-derived edges on top ── // LSP wins per edge — we union its arrays into ours. The LSP may know about // dynamic render names the scanner cannot see. for (const [path, lspEntry] of Object.entries(lspResults)) { @@ -231,13 +257,13 @@ function stripFilePrefix(uri) { return uri; } -// ── Dead code detection ────────────────────────────────────────────────────── +// ── Orphaned file detection ────────────────────────────────────────────────── /** - * Find files that nothing in the project references. A file is dead when its - * `referenced_by` is empty AND the file is not an entry point. + * Find files that nothing in the project references. A file is orphaned when + * its `referenced_by` is empty AND the file is not an entry point. * - * Entry points (never dead): + * Entry points (never orphaned): * - All pages (users reach them via HTTP routes). * - Files whose path contains /build/, /check/, or /execute/ — these are * phase files of multi-phase commands. Even when a parent command does @@ -249,13 +275,16 @@ function stripFilePrefix(uri) { * * @param {Record} graph * @param {object} projectMap - for entry-point classification (pages, etc.) - * @returns {string[]} dead file paths + * @returns {string[]} orphaned file paths */ -export function detectDeadCode(graph, projectMap) { +export function detectOrphanedFiles(graph, projectMap) { const dead = []; const pagePaths = new Set( Object.values(projectMap.pages ?? {}).map(p => p.path).filter(Boolean) ); + const layoutPaths = new Set( + Object.values(projectMap.layouts ?? {}).map(l => l.path).filter(Boolean) + ); for (const [path, edges] of Object.entries(graph)) { if ((edges.referenced_by ?? []).length > 0) continue; @@ -263,12 +292,23 @@ export function detectDeadCode(graph, projectMap) { // Pages are always live — they are HTTP entry points. if (pagePaths.has(path)) continue; + // Layouts are infrastructure — even unreferenced layouts may be used + // by pages via the default layout convention. If a layout IS referenced + // by pages (via page → layout edges) it won't reach here. If it's truly + // unused, it still shouldn't be auto-flagged: layout selection happens + // at runtime via frontmatter and the default layout fallback. + if (layoutPaths.has(path)) continue; + // Subphase command files are live by convention. if (isSubphaseFile(path)) continue; // GraphQL files: flagged elsewhere by the integrity check for orphaned ops. if (path.endsWith('.graphql')) continue; + // Translation files: structural errors are surfaced by translation-validator; + // they are never rendered by liquid files so they have no referenced_by edges. + if (path.startsWith('app/translations/')) continue; + // Anything outside app/ is assumed external (module fallback path) and skipped. if (!path.startsWith('app/')) continue; diff --git a/src/core/diagnostic-pipeline.js b/src/core/diagnostic-pipeline.js index fc4122c..bf89a6c 100644 --- a/src/core/diagnostic-pipeline.js +++ b/src/core/diagnostic-pipeline.js @@ -6,6 +6,10 @@ * and is documented with its purpose and ordering dependencies. * * ORDERING CONTRACT: + * 0a. suppressLspKnownFalsePositives — must run FIRST (after raw user suppressions) so + * downstream enrichment, fix generation, and the must_fix_before_write gate + * never see the spurious LSP error. Currently covers the pos-cli LSP + * "Syntax is not supported" regression on `assign x = a b`. * 1. suppressDocParams — must run before Shopify elevation (doc params may look like Shopify objects) * 2. suppressUnusedDocParams — depends on content, independent of other filters * 3. elevateShopify — must run after enrichment (needs .suggestion field) @@ -28,6 +32,15 @@ * 15. verifyOrphanedPartialOnDisk — independent (filesystem check) — must run AFTER 8 * so pending-plan suppression runs first; this catches the post-write case where * the files ARE on disk but the checker hasn't re-indexed (scaffold write:true). + * 16. verifyMissingPartialsOnDisk — independent (filesystem check) — must run AFTER 9 + * so pending suppression handles in-plan partials first; the disk check then + * catches partials that ARE on disk but the LSP hasn't re-indexed yet. + * 17. populateDefaultConfidence — must run LAST (after all suppressions/verifications) + * so it only stamps diagnostics that actually survive to the agent. The rule + * engine sets confidence and rule_id when a rule matches; this step covers + * everything else with a severity-based default confidence and a stable + * `${check}.unmatched` rule_id fallback (A4) so confidenceCalibration can bucket + * every row and the Rule Performance table attributes every emit to some rule. * * NOTE: MissingPartial, MissingPage and TranslationKeyExists are real errors — do NOT downgrade * them based on isPreWrite or other implicit state. Use pending_files / pending_pages / @@ -37,10 +50,13 @@ import { existsSync, readFileSync, readdirSync } from 'node:fs'; import { join } from 'node:path'; +import yaml from 'js-yaml'; +import { toLiquidHtmlAST } from '@platformos/liquid-html-parser'; import { getKnownModulesMissingDocs } from './knowledge-loader.js'; import { buildAssetIndex, resolveAssetPath } from './asset-index.js'; import { buildTranslationIndex } from './translation-index.js'; import { buildPageRouteIndex, parseMissingPageMessage, resolvePageRoute } from './page-route-index.js'; +import { DEFAULT_CONFIDENCE_BY_SEVERITY, STRUCTURAL_DEFAULT_CONFIDENCE } from './constants.js'; /** * Run the full diagnostic post-processing pipeline. @@ -66,80 +82,103 @@ export function runDiagnosticPipeline(result, opts) { projectDir, } = opts; + // Pipeline trace (D2 — pipeline step inspector). Each step records what changed. + const trace = []; + function traceStep(name, fn) { + const eBefore = result.errors.length; + const wBefore = result.warnings.length; + fn(); + const eRemoved = eBefore - result.errors.length; + const wRemoved = wBefore - result.warnings.length; + const eAdded = result.errors.length - (eBefore - eRemoved); + trace.push({ + step: name, + errorsRemoved: eRemoved, + warningsRemoved: wRemoved, + errorsAfter: result.errors.length, + warningsAfter: result.warnings.length, + }); + } + // Accumulate suppression summaries into one info diagnostic — the agent sees a single line. const suppressionNotes = []; + // 0. Apply user-defined suppressions from .pos-supervisor-ignore.yml (A3) + if (projectDir) { + traceStep('userSuppressions', () => applyUserSuppressions(result, filePath, projectDir)); + } + + // 0a. Suppress known pos-cli LSP false positives. Currently covers the + // "Syntax is not supported" regression on `assign x = a b` boolean + // comparisons. Runs first so downstream enrichment, fix generation, + // and the must_fix_before_write gate never see the spurious error. + traceStep('suppressLspKnownFalsePositives', () => suppressLspKnownFalsePositives(result, content)); + // 1. Suppress UndefinedObject for declared @param names if (docParamNames.size > 0) { - suppressDocParams(result, docParamNames); + traceStep('suppressDocParams', () => suppressDocParams(result, docParamNames)); } // 2. Suppress UnusedDocParam when param is used as named argument if (docParamNames.size > 0) { - suppressUnusedDocParams(result, docParamNames, content); + traceStep('suppressUnusedDocParams', () => suppressUnusedDocParams(result, docParamNames, content)); } // 3. Elevate Shopify contamination from warning to error - elevateShopify(result); + traceStep('elevateShopify', () => elevateShopify(result)); // 4. Deduplicate MissingRenderPartialArguments + MetadataParamsCheck - deduplicateArgChecks(result); + traceStep('deduplicateArgChecks', () => deduplicateArgChecks(result)); // 5. Suppress MetadataParamsCheck when the called target has no {% doc %} block. - // The LSP infers required params from usage patterns when no contract is declared, - // producing false positives for every optional param. Module partials (modules/*) - // are always treated as undocumented (they are excluded from lint by config AND - // overwhelmingly lack doc blocks in practice). App partials/commands/queries are - // confirmed by reading the target file from disk — if it has no {% doc %}, we - // suppress and emit an advisory info pointing at the root fix (add {% doc %}). - suppressUndocumentedTargetParams(result, content, projectDir); + traceStep('suppressUndocumentedTargetParams', () => suppressUndocumentedTargetParams(result, content, projectDir)); // 6. Suppress required-param diagnostics whose target partial defaults the param. - // The target's {% doc %} declared it required, but its body does `| default:`, - // so callers that omit the param still receive a valid value. This covers the - // common pattern where authors forgot to bracket the @param name. - suppressRequiredParamsWithDefault(result, content, projectDir); + traceStep('suppressRequiredParamsWithDefault', () => suppressRequiredParamsWithDefault(result, content, projectDir)); // 7. Suppress DeprecatedTag for module helper includes - suppressModuleHelpers(result, content); + traceStep('suppressModuleHelpers', () => suppressModuleHelpers(result, content)); - // 8. Suppress OrphanedPartial for commands/queries and for partials in - // multi-file creation plans (callers may be pending and not on disk yet). - suppressOrphanedPartial(result, filePath, pendingFiles, pendingPages); + // 8. Suppress OrphanedPartial for commands/queries and pending plans + traceStep('suppressOrphanedPartial', () => suppressOrphanedPartial(result, filePath, pendingFiles, pendingPages)); - // 8. Suppress MissingPartial for pending files + // 9. Suppress MissingPartial for pending files if (pendingFiles.length > 0) { - const n = suppressByPending(result, { - check: 'MissingPartial', - pendingSet: buildPendingPartialNames(pendingFiles), - extractKey: (d) => d.message?.match(/['"]([^'"]+)['"]/)?.[1] ?? null, + traceStep('suppressPendingPartials', () => { + const n = suppressByPending(result, { + check: 'MissingPartial', + pendingSet: buildPendingPartialNames(pendingFiles), + extractKey: (d) => d.message?.match(/['"]([^'"]+)['"]/)?.[1] ?? null, + }); + if (n > 0) suppressionNotes.push(`${n} MissingPartial(s) for pending files`); }); - if (n > 0) suppressionNotes.push(`${n} MissingPartial(s) for pending files`); } - // 9. Suppress MissingPage for pending pages + // 10. Suppress MissingPage for pending pages if (pendingPages.length > 0) { - const n = suppressByPending(result, { - check: 'MissingPage', - pendingSet: buildPendingPageKeys(pendingPages), - extractKey: (d) => { - // MissingPage messages look like: Page 'blog_posts/show' not found - // or: Missing page at slug 'blog_posts' - const m = d.message?.match(/['"]([^'"]+)['"]/); - return m ? m[1] : null; - }, + traceStep('suppressPendingPages', () => { + const n = suppressByPending(result, { + check: 'MissingPage', + pendingSet: buildPendingPageKeys(pendingPages), + extractKey: (d) => { + const m = d.message?.match(/['"]([^'"]+)['"]/); + return m ? m[1] : null; + }, + }); + if (n > 0) suppressionNotes.push(`${n} MissingPage(s) for pending pages`); }); - if (n > 0) suppressionNotes.push(`${n} MissingPage(s) for pending pages`); } - // 10. Suppress TranslationKeyExists for pending translations + // 11. Suppress TranslationKeyExists for pending translations if (pendingTranslations.length > 0) { - const n = suppressByPending(result, { - check: 'TranslationKeyExists', - pendingSet: new Set(pendingTranslations), - extractKey: (d) => d.message?.match(/['"]([^'"]+)['"]/)?.[1] ?? null, + traceStep('suppressPendingTranslations', () => { + const n = suppressByPending(result, { + check: 'TranslationKeyExists', + pendingSet: new Set(pendingTranslations), + extractKey: (d) => d.message?.match(/['"]([^'"]+)['"]/)?.[1] ?? null, + }); + if (n > 0) suppressionNotes.push(`${n} TranslationKeyExists for pending translations`); }); - if (n > 0) suppressionNotes.push(`${n} TranslationKeyExists for pending translations`); } if (suppressionNotes.length > 0) { @@ -150,44 +189,158 @@ export function runDiagnosticPipeline(result, opts) { }); } - // 11. Verify MissingAsset against filesystem + // 12. Verify MissingAsset against filesystem + if (projectDir) { + traceStep('verifyMissingAssets', () => verifyMissingAssets(result, projectDir)); + } + + // 13. Verify TranslationKeyExists against filesystem if (projectDir) { - verifyMissingAssets(result, projectDir); + traceStep('verifyTranslationKeysOnDisk', () => verifyTranslationKeysOnDisk(result, projectDir)); } - // 12. Verify TranslationKeyExists against filesystem. The LSP's translation - // cache lags behind disk just like its asset cache — after the agent - // writes a key to app/translations/.yml the LSP keeps reporting - // "key not found" until it re-indexes. Cross-check against the real - // YAML files so the agent does not need to pass `pending_translations` - // for keys that already exist on disk. + // 14. Verify MissingPage against filesystem. The file under validation is + // passed as an overlay so its in-memory frontmatter (`slug:`, `method:`) + // contributes to the route index — fixes the self-page case where an + // agent validating `app/views/pages/index.liquid` with `method: post` + // in-memory would otherwise see a stale MissingPage warning for the + // very route this page is about to serve. if (projectDir) { - verifyTranslationKeysOnDisk(result, projectDir); + traceStep('verifyPageRoutesOnDisk', () => verifyPageRoutesOnDisk(result, projectDir, { filePath, content })); } - // 13. Verify MissingPage against filesystem. validate_code analyses one - // file at a time, so any link in a partial pointing to a route defined - // in OTHER pages fires MissingPage. Cross-check against the real page - // files (slug from frontmatter or path-derived) so a header partial - // linking to /notes does not flag the route as missing when - // app/views/pages/notes/index.html.liquid clearly exists. + // 15. Verify OrphanedPartial against filesystem if (projectDir) { - verifyPageRoutesOnDisk(result, projectDir); + traceStep('verifyOrphanedPartialOnDisk', () => verifyOrphanedPartialOnDisk(result, filePath, projectDir)); } - // 14. Verify OrphanedPartial against filesystem. validate_code analyses - // one file at a time, so the checker has no cross-file render graph. - // After scaffold(write:true) writes all files and clears pending state, - // the checker still reports OrphanedPartial because its index hasn't - // re-indexed the new pages yet. Cross-check by scanning all .liquid - // files on disk for a render/function reference to this partial. + // 16. Verify MissingPartial against filesystem if (projectDir) { - verifyOrphanedPartialOnDisk(result, filePath, projectDir); + traceStep('verifyMissingPartialsOnDisk', () => verifyMissingPartialsOnDisk(result, projectDir)); } + + // 17. Stamp a default confidence on every surviving diagnostic that the rule + // engine did not already score. Runs last so suppressed/downgraded items + // are gone by now. + traceStep('populateDefaultConfidence', () => populateDefaultConfidence(result)); + + // Attach pipeline trace for dashboard inspector (D2) + result._pipelineTrace = trace; } // ── Individual filters ────────────────────────────────────────────────────── +/** + * Suppress known pos-cli LSP false positives. + * + * Currently covers ONE pattern: `LiquidHTMLSyntaxError` with the generic + * upstream message `Syntax is not supported`, fired by pos-cli's LSP on a + * boolean comparison inside an `assign` tag (e.g. `assign object.valid = c == empty`, + * `assign x = 1 == 1`). The platformOS Liquid parser + * (`@platformos/liquid-html-parser`, the same package the runtime parser is + * derived from) accepts this syntax, and `pos-cli check run` reports no + * offenses on it. Only the LSP rejects it — a stand-alone regression in the + * LSP's expression parser, not in the language. + * + * Without this suppression, agents are forced to rewrite valid Liquid + * (typically by lowering `assign x = a == b` to a four-line if/else) just to + * pass the must_fix_before_write gate. Worse, the diagnostic message + * "Syntax is not supported" gives no actionable hint, so agents iterate + * blindly until something passes. + * + * Suppression criteria — all must hold: + * 1. The diagnostic check is `LiquidHTMLSyntaxError`. + * 2. The message matches the exact upstream phrasing `Syntax is not supported` + * (case-insensitive). Other LiquidHTMLSyntaxError messages — unclosed + * blocks, malformed tag arguments, etc. — point at real bugs and stay. + * 3. The file parses cleanly under the strict mode of the platformOS parser. + * A genuine syntax error elsewhere in the file would fail strict parse, + * and we leave every diagnostic in place to avoid masking it. + * + * When all three hold, the matching diagnostics are removed and a single + * `pos-supervisor:LspSyntaxFalsePositiveSuppressed` info diagnostic is + * emitted naming the lines so the agent can audit the suppression. + */ +function suppressLspKnownFalsePositives(result, content) { + const matches = (d) => + d.check === 'LiquidHTMLSyntaxError' && + typeof d.message === 'string' && + /^Syntax is not supported$/i.test(d.message.trim()); + + const candidates = [ + ...result.errors.filter(matches), + ...result.warnings.filter(matches), + ]; + if (candidates.length === 0) return; + + // Strict parse — no tolerant flag — is the gate. If the platformOS parser + // rejects the file, there IS a genuine syntax error and we must not + // suppress LSP errors that may be the agent's only signal. + let parsesCleanly; + try { + toLiquidHtmlAST(content); + parsesCleanly = true; + } catch { + parsesCleanly = false; + } + if (!parsesCleanly) return; + + const removeSet = new Set(candidates); + result.errors = result.errors.filter(d => !removeSet.has(d)); + result.warnings = result.warnings.filter(d => !removeSet.has(d)); + + const lines = candidates + .map(d => d.line) + .filter(n => n != null); + result.infos.push({ + check: 'pos-supervisor:LspSyntaxFalsePositiveSuppressed', + severity: 'info', + message: + `Suppressed ${candidates.length} LiquidHTMLSyntaxError("Syntax is not supported") ` + + `diagnostic(s)${lines.length ? ` on line(s) ${lines.join(', ')}` : ''} — ` + + `the platformOS parser (@platformos/liquid-html-parser) accepts the file. ` + + `This is a known pos-cli LSP regression, most often triggered by a boolean ` + + `comparison inside \`assign\` (e.g. \`assign x = a == b\`).`, + }); +} + +function applyUserSuppressions(result, filePath, projectDir) { + const suppressFile = join(projectDir, '.pos-supervisor-ignore.yml'); + if (!existsSync(suppressFile)) return; + let rules; + try { + const parsed = yaml.load(readFileSync(suppressFile, 'utf-8')); + rules = parsed?.suppressions; + } catch { return; } + if (!Array.isArray(rules) || rules.length === 0) return; + + const matchRule = (d) => rules.some(r => { + if (r.check !== d.check) return false; + if (r.file_pattern) { + if (r.file_pattern.includes('*')) { + const re = new RegExp('^' + r.file_pattern.replace(/\*/g, '.*') + '$'); + if (!re.test(filePath)) return false; + } else if (!filePath.includes(r.file_pattern)) { + return false; + } + } + return true; + }); + + const errBefore = result.errors.length; + const warnBefore = result.warnings.length; + result.errors = result.errors.filter(d => !matchRule(d)); + result.warnings = result.warnings.filter(d => !matchRule(d)); + const suppressed = (errBefore - result.errors.length) + (warnBefore - result.warnings.length); + if (suppressed > 0) { + result.infos.push({ + check: 'pos-supervisor:UserSuppressed', + severity: 'info', + message: `Suppressed ${suppressed} diagnostic(s) via .pos-supervisor-ignore.yml`, + }); + } +} + function suppressDocParams(result, docParamNames) { const match = (diag) => { if (diag.check !== 'UndefinedObject') return false; @@ -755,11 +908,11 @@ function verifyMissingAssets(result, projectDir) { * link to a POST-only route) and the agent needs to see it. * - route not served at all → diagnostic stands. */ -function verifyPageRoutesOnDisk(result, projectDir) { +function verifyPageRoutesOnDisk(result, projectDir, currentFile = null) { const candidates = [...result.errors, ...result.warnings].filter(d => d.check === 'MissingPage'); if (candidates.length === 0) return; - const index = buildPageRouteIndex(projectDir); + const index = buildPageRouteIndex(projectDir, currentFile); if (index.routes.size === 0) return; const suppressed = new Set(); @@ -843,11 +996,186 @@ function verifyOrphanedPartialOnDisk(result, filePath, projectDir) { }); } +/** + * Cross-check every MissingPartial against the real filesystem. + * + * The LSP's partial index lags behind disk writes. A partial written during a + * scaffold step produces a false-positive MissingPartial until the LSP re-indexes. + * Module partials (names starting with 'modules/') are skipped — they are not local + * disk files and cannot be suppressed by presence checks. + */ +function verifyMissingPartialsOnDisk(result, projectDir) { + const candidates = [...result.errors, ...result.warnings].filter(d => d.check === 'MissingPartial'); + if (candidates.length === 0) return; + + const suppressed = new Set(); + const verified = []; + + for (const d of candidates) { + const nameMatch = d.message?.match(/['"]([^'"]+)['"]/); + if (!nameMatch) continue; + const name = nameMatch[1]; + if (name.startsWith('modules/')) continue; + + if (resolveMissingPartialPaths(name, projectDir).some(p => existsSync(p))) { + suppressed.add(d); + verified.push(name); + } + } + + if (suppressed.size === 0) return; + + result.errors = result.errors.filter(d => !suppressed.has(d)); + result.warnings = result.warnings.filter(d => !suppressed.has(d)); + result.infos.push({ + check: 'pos-supervisor:MissingPartialSuppressed', + severity: 'info', + message: `Suppressed ${verified.length} MissingPartial diagnostic(s) — partial(s) exist on disk: ${verified.join(', ')}. (LSP cache lag — partial was written but not yet re-indexed.)`, + }); +} + +/** + * Mirror upstream `DocumentsLocator` partial-resolution semantics: the + * `function` / `render` tags resolve relative to the partial search paths + * declared by `@platformos/platformos-common` — + * FILE_TYPE_DIRS[Partial] = ['views/partials', 'lib'] + * — joined under `app/`. So `commands/X` is found at `app/lib/commands/X.liquid` + * and `lib/commands/X` would only resolve at `app/lib/lib/commands/X.liquid` + * (which never exists in any sane project). DO NOT strip a leading `lib/` + * here: doing so silently suppresses the LSP's correct MissingPartial error + * for the invalid prefix and steers agents toward the bug. The `.html.liquid` + * variant is included for legacy projects whose partials use the layout + * extension; upstream itself only matches `.liquid`, so this is a superset. + */ +function resolveMissingPartialPaths(name, projectDir) { + return [ + join(projectDir, 'app', 'views', 'partials', `${name}.liquid`), + join(projectDir, 'app', 'views', 'partials', `${name}.html.liquid`), + join(projectDir, 'app', 'lib', `${name}.liquid`), + ]; +} + function extractPartialNameFromPath(filePath) { const m = filePath.match(/^app\/views\/partials\/(.+?)\.(?:html\.)?liquid$/); return m ? m[1] : null; } +function defaultConfidenceFor(diag) { + if (typeof diag.check === 'string' && diag.check.startsWith('pos-supervisor:')) { + return STRUCTURAL_DEFAULT_CONFIDENCE; + } + const sev = diag.severity; + if (sev && DEFAULT_CONFIDENCE_BY_SEVERITY[sev] != null) { + return DEFAULT_CONFIDENCE_BY_SEVERITY[sev]; + } + return DEFAULT_CONFIDENCE_BY_SEVERITY.warning; +} + +function defaultRuleIdFor(diag) { + // Stable fallback so rule-less diagnostics cluster under a single bucket per + // check instead of scattering into `unknown` or the check name alone (which + // collides with the check-level scorecard and muddles rule attribution). + // See A4 in docs/new-task/implementation-plan.md. + return diag.check ? `${diag.check}.unmatched` : 'unknown.unmatched'; +} + +function populateDefaultConfidence(result) { + const stamp = (d) => { + if (d.confidence == null) d.confidence = defaultConfidenceFor(d); + if (!d.rule_id) d.rule_id = defaultRuleIdFor(d); + }; + for (const d of result.errors) stamp(d); + for (const d of result.warnings) stamp(d); + for (const d of result.infos) stamp(d); +} + +/** + * Stand-alone entry point — same semantics as the pipeline's step 17, callable + * from outside the pipeline. + * + * Reason it exists: `validate-code.js` pushes several diagnostic sources + * (structural warnings, schema validation, translation YAML check, diff-aware + * RemovedRender/RemovedGraphQL/AddedParam, new-partial caller check) into + * `result.errors` / `result.warnings` AFTER `runDiagnosticPipeline` finishes. + * Those late additions would otherwise escape `populateDefaultConfidence` and + * land in the analytics store with `confidence = null` / `rule_id` missing. + * See the confidence-stamp bug identified in the 2026-04-23 DEMO report. + * + * Idempotent — calling twice is safe because the helper only fills when + * fields are null/missing. + */ +export function stampDefaultsOn(result) { + populateDefaultConfidence(result); +} + +/** + * Suppress upstream `ValidFrontmatter` diagnostics that overlap with our + * richer structural-check counterparts. pos-cli 6.0.7 added `ValidFrontmatter` + * which independently reports the same root causes as our existing + * `pos-supervisor:InvalidLayout` (missing layout file) and + * `pos-supervisor:InvalidFrontMatter` (unknown / misleading frontmatter keys). + * + * Our checks carry richer messages (named expected paths, deprecation + * guidance, fix templates) so we keep them and drop the upstream copy. + * Upstream `ValidFrontmatter` rows that don't share a line with one of our + * checks pass through untouched — they cover novel cases (deprecated + * `layout_name`, missing required fields per file type, invalid HTTP method + * enum, etc.) that our structural checks don't handle yet. + * + * Line-anchored: YAML frontmatter is one key per line, so a line collision + * between `ValidFrontmatter` and `pos-supervisor:InvalidLayout` / + * `pos-supervisor:InvalidFrontMatter` reliably indicates the same root cause. + * + * Idempotent. Pure. Safe to call after both diagnostic sources have pushed + * (i.e. after `generateStructuralWarnings`). + * + * @returns {number} count of suppressed diagnostics + */ +export function suppressUpstreamFrontmatterDup(result) { + // Two matching axes — line (the default) AND layout name (parsed from the + // message). The line-match alone misses cases where upstream and our + // structural emitter disagree by ±1 line (frontmatter edge cases, leading + // whitespace, line-zero anchoring), which is exactly what the DEMO data + // showed: both `pos-supervisor:InvalidLayout` and + // `ValidFrontmatter.layout_missing` fired with diverging line values, so + // the agent saw two contradictory hints for the same root cause. + const ourLines = new Set(); + const ourInvalidLayoutNames = new Set(); + for (const d of [...result.errors, ...result.warnings]) { + if (d.check === 'pos-supervisor:InvalidLayout' || d.check === 'pos-supervisor:InvalidFrontMatter') { + ourLines.add(d.line); + } + if (d.check === 'pos-supervisor:InvalidLayout') { + const layoutName = d.message?.match(/^Layout `([^`]+)` not found/)?.[1]; + if (layoutName) ourInvalidLayoutNames.add(layoutName); + } + } + if (ourLines.size === 0 && ourInvalidLayoutNames.size === 0) return 0; + + const isRedundant = (d) => { + if (d.check !== 'ValidFrontmatter') return false; + if (ourLines.has(d.line)) return true; + // Layout-name match: the upstream `Layout 'X' does not exist` shape + // names the same layout `X` that pos-supervisor:InvalidLayout already + // flagged. The two diagnostics describe identical root cause. + const layoutName = d.message?.match(/^Layout ['"`]([^'"`]+)['"`] does not exist$/)?.[1]; + return !!layoutName && ourInvalidLayoutNames.has(layoutName); + }; + const eRemoved = result.errors.filter(isRedundant).length; + const wRemoved = result.warnings.filter(isRedundant).length; + const removed = eRemoved + wRemoved; + if (removed === 0) return 0; + + result.errors = result.errors.filter(d => !isRedundant(d)); + result.warnings = result.warnings.filter(d => !isRedundant(d)); + result.infos.push({ + check: 'pos-supervisor:DuplicateFrontmatterCheck', + severity: 'info', + message: `Suppressed ${removed} ValidFrontmatter diagnostic(s) already covered by pos-supervisor structural check(s) (InvalidLayout / InvalidFrontMatter).`, + }); + return removed; +} + function hasRenderReferenceOnDisk(projectDir, partialName, selfPath) { const escaped = partialName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); const pattern = new RegExp(`['"]${escaped}['"]`); diff --git a/src/core/diagnostic-record.js b/src/core/diagnostic-record.js new file mode 100644 index 0000000..74af617 --- /dev/null +++ b/src/core/diagnostic-record.js @@ -0,0 +1,392 @@ +/** + * DiagnosticRecord — typed, fingerprinted, projection-friendly representation + * of one warning or error from the LSP / structural / symbolic layer. + * + * Why this exists (roadmap §3.1, gap #3 + #4): + * - Today every consumer (error-enricher, fix-generator, analyze-project) + * re-parses the human-readable LSP `message` with its own regex to pull + * out a partial name / variable name / translation key. Those regexes + * drift, the boundary explodes whenever the LSP message format shifts, + * and analytics has no stable identity to track resolution across calls. + * - DiagnosticRecord builds the structured form ONCE at the layer that + * produced the diagnostic. Downstream code reads `diag.params.partial` + * instead of grepping a string. Fingerprints (`fp`, `template_fp`) give + * the analytics layer stable identifiers. + * + * Stability contract: + * - `template_fp = sha1(check + ":" + messageTemplate)` — same across files, + * locales, and instances. Used for "how often does this *kind* of error + * fire?" cross-file analytics. + * - `fp = sha1(check + ":" + file + ":" + messageTemplate)` — same record + * across calls if the file/check/template don't change. Used to detect + * "has this exact diagnostic been resolved?" between consecutive + * validate_code calls. + * - Both EXCLUDE position fields. A page edit that shifts a diagnostic from + * line 12 to line 14 is still the same diagnostic; the fingerprint must + * not flip. + * - Pinning test: tests/upstream/diagnostic-fingerprint.test.js. If you + * change the masking algorithm or the param extractor for a check, + * the pin will scream — that's intentional, the schema bumps. + */ + +import { createHash } from 'node:crypto'; + +export const DIAGNOSTIC_RECORD_VERSION = 1; + +/** + * Build a DiagnosticRecord from a raw diagnostic produced by the LSP, the + * structural-warnings layer, or a future symbolic-rule engine. + * + * @param {object} raw - { check, severity, message, line, column, end_line?, end_character? } + * @param {object} opts + * @param {string} opts.file - Project-relative path (NOT a file:// uri). + * @param {'lsp'|'structural'|'symbolic'} opts.source + * @param {object} [opts.origin] - { check_runner_version?, lsp_version? } + * @returns {object} frozen DiagnosticRecord + */ +export function makeDiagnosticRecord(raw, { file, source, origin = {} } = {}) { + if (!raw || typeof raw !== 'object') { + throw new Error('makeDiagnosticRecord: raw diagnostic required'); + } + if (!raw.check) throw new Error('makeDiagnosticRecord: raw.check required'); + if (!file) throw new Error('makeDiagnosticRecord: file required'); + if (!source) throw new Error('makeDiagnosticRecord: source required'); + + const message = typeof raw.message === 'string' ? raw.message : ''; + const messageTemplate = templateOf(raw.check, message); + const params = extractParams(raw.check, message); + + const position = { + line: numberOr(raw.line, 0), + character: numberOr(raw.column, 0), + end_line: numberOr(raw.end_line, numberOr(raw.line, 0)), + end_character: numberOr(raw.end_character, numberOr(raw.column, 0)), + }; + + return Object.freeze({ + v: DIAGNOSTIC_RECORD_VERSION, + fp: fingerprint(raw.check, file, messageTemplate), + template_fp: templateFingerprint(raw.check, messageTemplate), + check: raw.check, + severity: normalizeSeverity(raw.severity), + file, + position: Object.freeze(position), + message, + message_template: messageTemplate, + params: Object.freeze(params), + source, + origin: Object.freeze({ + check_runner_version: origin.check_runner_version ?? null, + lsp_version: origin.lsp_version ?? null, + }), + }); +} + +// ── Fingerprint helpers ────────────────────────────────────────────────────── + +export function fingerprint(check, file, messageTemplate) { + return sha1(`${check}:${file}:${messageTemplate}`); +} + +export function templateFingerprint(check, messageTemplate) { + return sha1(`${check}:${messageTemplate}`); +} + +function sha1(input) { + return createHash('sha1').update(input, 'utf8').digest('hex'); +} + +// ── Message template (identifier mask) ─────────────────────────────────────── +// +// Goal: two diagnostics with the same SHAPE but different identifiers (file +// names, variable names, translation keys) collapse to the same template +// string. The mask is intentionally minimal — we replace only the things +// that vary across instances of the same check, never things that +// discriminate distinct check shapes. +// +// Substitutions, in order: +// 1. Quoted identifiers ('x', "x", `x`) → +// 2. Bare ASCII numbers (12, 0xff, 1.5) → +// 3. Run-of-whitespace → single space +// 4. Trim leading/trailing whitespace +// +// We do NOT lowercase — case can be load-bearing in some checks. + +export function messageTemplate(message) { + if (typeof message !== 'string' || message === '') return ''; + let out = message; + // Quoted strings (single, double, backtick). Greedy stops at the next + // matching quote without crossing newlines so multi-message blobs survive. + out = out.replace(/`([^`\n]*)`/g, ''); + out = out.replace(/'([^'\n]*)'/g, ''); + out = out.replace(/"([^"\n]*)"/g, ''); + // Bare numbers (decimal, hex, float). Word-boundary anchored so we don't + // chew "v1" → "v" or "html5" → "html". + out = out.replace(/\b\d+(?:\.\d+)?\b/g, ''); + out = out.replace(/\b0x[0-9a-fA-F]+\b/g, ''); + out = out.replace(/\s+/g, ' ').trim(); + return out; +} + +// Per-check template override hook. Today the generic mask is sufficient for +// every check we ship. If a check's message format ever requires a custom +// mask (e.g. the LSP starts emitting timestamps inside the message), wire it +// here rather than hard-coding inside the generic mask. +const TEMPLATE_OVERRIDES = Object.freeze({ + // Example shape — keep commented as a contract for future authors: + // 'GraphQLCheck': (msg) => msg.replace(/at line \d+/, 'at line '), +}); + +export function templateOf(check, message) { + const override = TEMPLATE_OVERRIDES[check]; + return override ? override(messageTemplate(message)) : messageTemplate(message); +} + +// ── Param extraction (typed, per check) ────────────────────────────────────── +// +// Each extractor returns a flat string-keyed object. The `params` field on +// DiagnosticRecord is the contract downstream consumers code against — +// changes here are observable to enrichers, fix generators, and analytics. + +const QUOTED = /[`'"]([^`'"]+)[`'"]/; + +function firstQuoted(message) { + const m = message.match(QUOTED); + return m ? m[1] : null; +} + +function pairQuoted(message) { + // Two distinct quoted spans, in order of appearance. + const bt = message.match(/`([^`]+)`[^`]*`([^`]+)`/); + const dq = message.match(/"([^"]+)"[^"]*"([^"]+)"/); + const sq = message.match(/'([^']+)'[^']*'([^']+)'/); + const m = bt || dq || sq; + return m ? [m[1], m[2]] : null; +} + +const EXTRACTORS = Object.freeze({ + UnknownFilter(message) { + const filter = firstQuoted(message); + return filter ? { filter } : {}; + }, + + UndefinedObject(message) { + // LSP message: "The object 'foo' is undefined". Earlier the codebase + // used a dedicated extractVarName regex; we keep the same first-quoted + // contract so existing tests pass without touching the enricher yet. + const variable = firstQuoted(message); + return variable ? { variable } : {}; + }, + + UnusedAssign(message) { + const variable = firstQuoted(message); + return variable ? { variable } : {}; + }, + + MissingPartial(message) { + const partial = firstQuoted(message); + return partial ? { partial } : {}; + }, + + TranslationKeyExists(message) { + const key = firstQuoted(message); + if (!key) return {}; + const params = { key }; + if (/did you mean/i.test(message)) params.has_typo_suggestion = 'true'; + return params; + }, + + UnknownProperty(message) { + const pair = pairQuoted(message); + if (!pair) return {}; + return { property: pair[0], object: pair[1] }; + }, + + DeprecatedTag(message) { + // Tag is the first identifier; replacement (when present) follows + // "replaced by" or "use". + const tag = (message.match(/[`'"](\w+)[`'"]/) || message.match(/\btag\s+[`'"]?(\w+)[`'"]?/i) || [])[1] ?? null; + const replMatch = message.match(/replaced\s+by\s+\[?[`'"](\w+)[`'"]\]?/i) + || message.match(/\buse\s+[`'"](\w+)[`'"]/i); + const replacement = replMatch ? replMatch[1] : (tag === 'include' ? 'render' : null); + const params = {}; + if (tag) params.tag = tag; + if (replacement) params.replacement = replacement; + return params; + }, + + MissingRenderPartialArguments(message) { + const partialMatch = message.match(/[`'"]([^`'"]+\/[^`'"]+)[`'"]/); + const paramMatch = message.match(/\bargument\s+['"`](\w+)['"`]/i); + const params = {}; + if (partialMatch) params.partial = partialMatch[1]; + if (paramMatch) params.missing_param = paramMatch[1]; + return params; + }, + + MetadataParamsCheck(message) { + return { is_function_call: /function call/i.test(message) ? 'true' : 'false' }; + }, + + PartialCallArguments(message) { + // Two distinct LSP message shapes: + // "Required parameter must be passed to (render|function|GraphQL) call" + // "Unknown parameter passed to (render|function|GraphQL) call" + // Both carry the param name and the call kind; neither carries the + // partial / function path (the sibling MissingRenderPartialArguments + // does, when the agent is rendering a partial). + const requiredMatch = message.match(/^Required parameter\s+([A-Za-z_][\w]*)\s+must be passed to (\w+)\s+call/i); + const unknownMatch = message.match(/^Unknown parameter\s+([A-Za-z_][\w]*)\s+passed to (\w+)\s+call/i); + const m = requiredMatch || unknownMatch; + if (!m) return {}; + const callKind = m[2].toLowerCase(); + return { + param_name: m[1], + direction: requiredMatch ? 'required' : 'unknown', + call_kind: callKind, // 'render' | 'function' | 'graphql' + is_function_call: callKind === 'function' ? 'true' : 'false', + }; + }, + + GraphQLVariablesCheck(message) { + // LSP shape mirrors PartialCallArguments but always carries the + // GraphQL call kind. We surface the same params so a generic param- + // mismatch handler (rule layer) can route on `direction` + the + // operation name once we plumb it. + const requiredMatch = message.match(/^Required parameter\s+([A-Za-z_][\w]*)\s+must be passed to GraphQL call/i); + const unknownMatch = message.match(/^Unknown parameter\s+([A-Za-z_][\w]*)\s+passed to GraphQL call/i); + const m = requiredMatch || unknownMatch; + if (!m) return {}; + return { + param_name: m[1], + direction: requiredMatch ? 'required' : 'unknown', + call_kind: 'graphql', + }; + }, + + UnusedDocParam(message) { + // LSP shape: "The parameter 'name' is defined but not used in this file." + const m = message.match(/^The parameter\s+['"`]([A-Za-z_][\w]*)['"`]\s+is defined but not used/i); + return m ? { param_name: m[1] } : {}; + }, + + ValidFrontmatter(message) { + // pos-cli 6.0.7 ships a single check that emits eight distinct shapes. + // We classify into a `category` so the rule engine can route to a + // category-specific hint without re-parsing the message itself. + // + // Categories: + // home_deprecated, missing_required, unknown_field, deprecated_field, + // invalid_enum, layout_false, layout_missing, association_missing, + // unknown (fallback — surfaces as `.unmatched` if it ever fires). + if (/'home\.html\.liquid' is deprecated/i.test(message)) { + return { category: 'home_deprecated' }; + } + let m = message.match(/^Missing required frontmatter field [`'"]([^`'"]+)[`'"] in (.+?) file$/); + if (m) return { category: 'missing_required', field: m[1], file_type: m[2] }; + m = message.match(/^Unknown frontmatter field [`'"]([^`'"]+)[`'"] in (.+?) file$/); + if (m) return { category: 'unknown_field', field: m[1], file_type: m[2] }; + if (/^`layout: false`/.test(message)) return { category: 'layout_false' }; + m = message.match(/^Layout [`'"]([^`'"]+)[`'"] does not exist$/); + if (m) return { category: 'layout_missing', layout: m[1] }; + m = message.match(/^Invalid value [`'"]([^`'"]+)[`'"] for [`'"]([^`'"]+)[`'"]\. Must be one of: (.+)$/); + if (m) return { category: 'invalid_enum', value: m[1], field: m[2], allowed: m[3] }; + m = message.match(/^[`'"]([^`'"]+)[`'"] is deprecated/); + if (m) return { category: 'deprecated_field', field: m[1] }; + if (/deprecated/i.test(message)) { + // Custom deprecation messages from per-field schemas — extract the first + // quoted token as a best-effort field hint. + const f = firstQuoted(message); + return f ? { category: 'deprecated_field', field: f } : { category: 'deprecated_field' }; + } + m = message.match(/^(.+?) [`'"]([^`'"]+)[`'"] does not exist$/); + if (m) return { category: 'association_missing', label: m[1], name: m[2] }; + return { category: 'unknown' }; + }, + + JsonLiteralQuoteStyle(_message) { + // Single-shot message — no params extracted. The category is implicit + // (always "single quote inside JSON literal"). Returning {} keeps the + // bag JSON-safe and lets the rule engine fire on `check` alone. + return {}; + }, + + DuplicateFunctionArguments(message) { + // "Duplicate argument 'x' in render tag for partial 'p'." + // "Duplicate argument 'x' in function tag for partial 'p'." + const m = message.match(/^Duplicate argument [`'"]([^`'"]+)[`'"] in (\w+) tag for partial [`'"]([^`'"]+)[`'"]\.?$/); + if (m) return { argument: m[1], tag_kind: m[2], partial: m[3] }; + return {}; + }, + + GraphQLCheck(message) { + const unused = message.match(/Variable\s+["']?\$(\w+)["']?\s+is never used/i); + if (unused) return { category: 'unused_variable', variable: unused[1] }; + + const fieldMatch = message.match(/Cannot query field\s+["']?(\w+)["']?\s+on type\s+["']?(\w+)["']?/i); + if (fieldMatch) { + return { + category: fieldMatch[2] === 'Record' ? 'unknown_field_record' : 'unknown_field_other', + field: fieldMatch[1], + type: fieldMatch[2], + }; + } + + const typeMismatch = message.match(/Variable\s+["']?\$(\w+)["']?\s+of type\s+["']?([^"']+)["']?\s+used in position expecting(?: type)?\s+["']?([^"'.]+)["']?/i); + if (typeMismatch) { + const expected = typeMismatch[3].trim(); + return { + category: /filter/i.test(expected) ? 'type_mismatch_filter' : 'type_mismatch_other', + variable: typeMismatch[1], + actual_type: typeMismatch[2], + expected_type: expected, + }; + } + + const filterMatch = message.match(/Expected value of type\s+["']?(\w+)["']?,?\s+found\s+["']?([^"'.]+)["']?/i); + if (filterMatch) { + return { + category: /filter/i.test(filterMatch[1]) ? 'type_mismatch_filter' : 'type_mismatch_other', + actual_type: `"${filterMatch[2].trim()}"`, + expected_type: filterMatch[1], + }; + } + + return { category: 'generic' }; + }, +}); + +export function extractParams(check, message) { + const fn = EXTRACTORS[check]; + if (!fn) return {}; + try { + const out = fn(message ?? ''); + // Defensive: stringify everything so the params bag is JSON-safe and the + // analytics SQL layer (Phase B) can index without column-type drift. + const safe = {}; + for (const [k, v] of Object.entries(out)) { + if (v == null) continue; + safe[k] = typeof v === 'string' ? v : String(v); + } + return safe; + } catch { + return {}; + } +} + +export const KNOWN_EXTRACTOR_CHECKS = Object.freeze(Object.keys(EXTRACTORS)); + +// ── Internal helpers ───────────────────────────────────────────────────────── + +function numberOr(v, fallback) { + return Number.isFinite(v) ? v : fallback; +} + +function normalizeSeverity(s) { + if (s === 'error' || s === 'warning' || s === 'info') return s; + // LSP DiagnosticSeverity codes (1-4) sometimes leak through. + if (s === 1) return 'error'; + if (s === 2) return 'warning'; + if (s === 3 || s === 4) return 'info'; + return 'warning'; +} diff --git a/src/core/engine-mode.js b/src/core/engine-mode.js new file mode 100644 index 0000000..a8bdadb --- /dev/null +++ b/src/core/engine-mode.js @@ -0,0 +1,85 @@ +/** + * Engine mode — global toggle for the neuro-symbolic write side. + * + * Two modes: + * 'adaptive' — auto-disabling, case-base scoring, promoted rules, probation + * 'static' — all rules fire at raw confidence, no promoted rules loaded + * + * Analytics data collection runs in BOTH modes. Only the consumption + * of analytics (scoring, disabling, promoting) is gated. + */ + +import { readFileSync, writeFileSync, mkdirSync, existsSync } from 'node:fs'; +import { join } from 'node:path'; + +const MODE_FILE = 'engine-mode.json'; +const VALID_MODES = new Set(['adaptive', 'static']); + +let _mode = 'static'; +let _listeners = []; + +export function getEngineMode() { + return _mode; +} + +export function isAdaptive() { + return _mode === 'adaptive'; +} + +export function setEngineMode(mode, { projectDir, onTransition } = {}) { + if (!VALID_MODES.has(mode)) { + throw new Error(`Invalid engine mode: '${mode}'. Must be 'adaptive' or 'static'.`); + } + const prev = _mode; + if (prev === mode) return; + + _mode = mode; + + if (projectDir) { + persistEngineMode(projectDir, mode); + } + + if (onTransition) { + onTransition(prev, mode); + } + + for (const fn of _listeners) { + try { fn(mode, prev); } catch { /* listener failure is non-fatal */ } + } +} + +export function onEngineModeChange(fn) { + _listeners.push(fn); + return () => { + _listeners = _listeners.filter(f => f !== fn); + }; +} + +export function loadEngineMode(projectDir) { + const filePath = join(projectDir, '.pos-supervisor', MODE_FILE); + if (!existsSync(filePath)) return _mode; + + try { + const raw = JSON.parse(readFileSync(filePath, 'utf-8')); + if (VALID_MODES.has(raw?.mode)) { + _mode = raw.mode; + } + } catch { /* malformed file — keep current mode */ } + + return _mode; +} + +export function persistEngineMode(projectDir, mode) { + const dir = join(projectDir, '.pos-supervisor'); + if (!existsSync(dir)) mkdirSync(dir, { recursive: true }); + writeFileSync( + join(dir, MODE_FILE), + JSON.stringify({ mode, updated_at: new Date().toISOString() }, null, 2) + '\n', + 'utf-8', + ); +} + +export function resetEngineMode() { + _mode = 'static'; + _listeners = []; +} diff --git a/src/core/error-enricher.js b/src/core/error-enricher.js index 41696f4..43a9630 100644 --- a/src/core/error-enricher.js +++ b/src/core/error-enricher.js @@ -1,6 +1,7 @@ import { getHint } from './hint-loader.js'; -import { extractVarName } from './objects-index.js'; +import { extractParams, templateOf } from './diagnostic-record.js'; import { isShopifyObject, isShopifyFilter, getShopifyObject, getShopifyFilter } from './knowledge-loader.js'; +import { runRules, hasRules } from './rules/engine.js'; /** * Extract readable text from LSP hover result. @@ -27,7 +28,7 @@ function extractHoverText(result) { * @param {object} ctx.schemaIndex * @returns {Promise} Enriched diagnostic */ -export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsIndex, tagsIndex, schemaIndex, content, _hoverCache }) { +export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsIndex, tagsIndex, schemaIndex, analyticsStore, content, _hoverCache, factGraph, filePath, projectDir }) { const result = { ...diagnostic }; // 1. Hint set per-check below with template vars; fallback for unhandled checks at end @@ -50,9 +51,31 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } } + // 2b. Rule engine — when rules exist for this check and a fact graph is + // available, run rules first. If a rule matches, use its output for + // hint/see_also/rule_id and skip the regex-based enrichment below. + if (factGraph && hasRules(diagnostic.check)) { + const params = extractParams(diagnostic.check, diagnostic.message); + const tmplFp = templateOf(diagnostic.check, diagnostic.message); + const diag = { check: diagnostic.check, params, message: diagnostic.message, file: filePath, line: diagnostic.line, column: diagnostic.column ?? 0, template_fp: tmplFp }; + const facts = { graph: factGraph, filtersIndex, objectsIndex, tagsIndex, schemaIndex, analyticsStore, projectDir }; + const ruleResult = runRules(diag, facts); + if (ruleResult) { + result.hint = ruleResult.hint_md; + result.rule_id = ruleResult.rule_id; + if (ruleResult.suggestion) result.suggestion = ruleResult.suggestion; + if (ruleResult.see_also) result.see_also = ruleResult.see_also; + if (ruleResult.confidence != null) result.confidence = ruleResult.confidence; + if (ruleResult.case_base_signal) result.case_base_signal = ruleResult.case_base_signal; + if (ruleResult.fixes?.length > 0) result.fixes = ruleResult.fixes; + attachSeeAlso(result, content); + return result; + } + } + // 3. Index lookup for suggestions + Shopify awareness if (diagnostic.check === 'UnknownFilter') { - const filterName = extractFilterName(diagnostic.message); + const filterName = extractParams(diagnostic.check, diagnostic.message).filter ?? null; let suggestion = null; if (filterName) { if (tagsIndex?.isTag(filterName)) { @@ -85,7 +108,7 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } if (diagnostic.check === 'UndefinedObject') { - const varName = extractVarName(diagnostic.message); + const varName = extractParams(diagnostic.check, diagnostic.message).variable ?? null; const isPartial = uri?.includes('/partials/'); // Compute suggestion first so has_suggestion can be passed to hint template let suggestion = null; @@ -123,9 +146,9 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } if (diagnostic.check === 'TranslationKeyExists') { - const key = extractTranslationKey(diagnostic.message); - // Check if the linter message already contains a typo suggestion ("Did you mean...") - const hasSuggestion = key && /did you mean/i.test(diagnostic.message); + const _tp = extractParams(diagnostic.check, diagnostic.message); + const key = _tp.key ?? null; + const hasSuggestion = _tp.has_typo_suggestion === 'true'; if (key) { result.hint = getHint(diagnostic.check, null, { key, @@ -137,11 +160,13 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } if (diagnostic.check === 'MissingPartial') { - const partialName = extractPartialName(diagnostic.message); + const partialName = extractParams(diagnostic.check, diagnostic.message).partial ?? null; const objType = detectObjectType(partialName); const createPath = buildCreatePath(objType, partialName); const tag = objType === 'partial' ? 'render' : 'function'; - const hintVariant = objType === 'module' ? 'module' : null; + let hintVariant = null; + if (objType === 'module') hintVariant = 'module'; + else if (objType === 'invalid_lib_prefix') hintVariant = 'invalid_lib_prefix'; // For module paths: fetch LSP completions to show available paths. // For project paths: agent has project_map context — no completions needed. @@ -174,6 +199,9 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } if (suggestion) result.suggestion = suggestion; + const correctedName = objType === 'invalid_lib_prefix' + ? partialName.slice('lib/'.length) + : null; result.hint = partialName ? getHint(diagnostic.check, hintVariant, { object: objType, @@ -181,12 +209,13 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI create_path: createPath, tag, has_suggestion: !!suggestion, + ...(correctedName ? { corrected_name: correctedName } : {}), }) : getHint(diagnostic.check, hintVariant); } if (diagnostic.check === 'UnknownProperty') { - const { propertyName, objectName } = extractPropertyAndObject(diagnostic.message); + const { property: propertyName = null, object: objectName = null } = extractParams(diagnostic.check, diagnostic.message); const propVariant = uri?.includes('/partials/') ? 'partial' : null; result.hint = (propertyName && objectName) ? getHint(diagnostic.check, propVariant, { @@ -197,7 +226,7 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } if (diagnostic.check === 'DeprecatedTag') { - const { tagName, replacementTag } = extractDeprecatedTagInfo(diagnostic.message); + const { tag: tagName = null, replacement: replacementTag = null } = extractParams(diagnostic.check, diagnostic.message); result.hint = tagName ? getHint(diagnostic.check, null, { tag_name: tagName, @@ -222,7 +251,7 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } if (diagnostic.check === 'MissingRenderPartialArguments') { - const { partialName, missingParam } = extractMissingArgInfo(diagnostic.message); + const { partial: partialName = null, missing_param: missingParam = null } = extractParams(diagnostic.check, diagnostic.message); result.hint = (partialName || missingParam) ? getHint(diagnostic.check, null, { partial_name: partialName ?? 'unknown', @@ -240,7 +269,7 @@ export async function enrichError(diagnostic, { uri, lsp, filtersIndex, objectsI } if (diagnostic.check === 'UnusedAssign') { - const varName = extractVarName(diagnostic.message); + const varName = extractParams(diagnostic.check, diagnostic.message).variable ?? null; result.hint = varName ? getHint(diagnostic.check, null, { var_name: varName }) : getHint(diagnostic.check, null); @@ -416,66 +445,10 @@ function classifyGraphQLError(message) { return { category_generic: true }; } -/** - * Extract property name and object name from an UnknownProperty error message. - * Handles: "Unknown property 'foo' on 'bar'", "property `foo` ... `bar`", etc. - */ -function extractPropertyAndObject(message) { - if (!message) return { propertyName: null, objectName: null }; - const bt = message.match(/`([^`]+)`[^`]*`([^`]+)`/); - const dq = message.match(/"([^"]+)"[^"]*"([^"]+)"/); - const sq = message.match(/'([^']+)'[^']*'([^']+)'/); - const m = bt || dq || sq; - return m ? { propertyName: m[1], objectName: m[2] } : { propertyName: null, objectName: null }; -} - -/** - * Extract tag name and replacement from a DeprecatedTag error message. - */ -function extractDeprecatedTagInfo(message) { - if (!message) return { tagName: null, replacementTag: null }; - const tagMatch = message.match(/[`'"](\w+)[`'"]/) || message.match(/\btag\s+[`'"]?(\w+)[`'"]?/i); - const tagName = tagMatch ? tagMatch[1] : null; - // Match "replaced by `render`" or "use `render`" — but NOT "use the way" or "reduces" - const replMatch = message.match(/replaced\s+by\s+\[?[`'"](\w+)[`'"]\]?/i) - || message.match(/\buse\s+[`'"](\w+)[`'"]/i); - const replacementTag = replMatch ? replMatch[1] : (tagName === 'include' ? 'render' : null); - return { tagName, replacementTag }; -} - -/** - * Extract partial name and missing param from a MissingRenderPartialArguments error message. - */ -function extractMissingArgInfo(message) { - if (!message) return { partialName: null, missingParam: null }; - // Partial name: quoted path containing a slash, e.g. 'products/card' - const partialMatch = message.match(/[`'"]([^`'"]+\/[^`'"]+)[`'"]/); - const partialName = partialMatch ? partialMatch[1] : null; - // Parameter name: matches "argument 'name'" in the actual linter message format: - // "Missing required argument 'email' in render tag for partial 'sessions/form'" - const paramMatch = message.match(/\bargument\s+['"`](\w+)['"`]/i); - const missingParam = paramMatch ? paramMatch[1] : null; - return { partialName, missingParam }; -} - -/** - * Extract filter name from an UnknownFilter error message. - */ -function extractFilterName(message) { - if (!message) return null; - const m = message.match(/`([^`]+)`/) || message.match(/"([^"]+)"/) || message.match(/'([^']+)'/); - return m ? m[1] : null; -} - -/** - * Extract translation key from a TranslationKeyExists error message. - * Message format: "Translation key 'some.key' not found." or similar. - */ -function extractTranslationKey(message) { - if (!message) return null; - const m = message.match(/['"`]([^'"`]+)['"`]/); - return m ? m[1] : null; -} +// Extraction functions (extractFilterName, extractTranslationKey, +// extractPartialName, extractPropertyAndObject, extractDeprecatedTagInfo, +// extractMissingArgInfo) have been centralized into diagnostic-record.js +// extractParams(). See roadmap §A2. /** * Build an indented YAML snippet showing where to add a translation key. @@ -496,14 +469,6 @@ function buildYamlSnippet(key) { return lines.join('\n'); } -/** - * Extract partial name from a MissingPartial error message. - */ -function extractPartialName(message) { - if (!message) return null; - const m = message.match(/['"]([^'"]+)['"]/); - return m ? m[1] : null; -} /** * Normalize LSP completion result to an array of label strings. @@ -523,14 +488,22 @@ function extractCompletionLabels(result) { function detectObjectType(name) { if (!name) return 'partial'; if (name.startsWith('modules/')) return 'module'; - if (/(?:^|\/)commands\//.test(name)) return 'command'; - if (/(?:^|\/)queries\//.test(name)) return 'query'; + // Literal `lib/commands/` or `lib/queries/` prefix is invalid: `function` + // tag paths resolve under the partial search paths, so `lib/commands/X` + // expands to `app/lib/lib/commands/X` which never exists. Tag separately + // so the hint renderer can surface "drop the prefix" instead of the + // generic "missing file" copy. + if (name.startsWith('lib/commands/') || name.startsWith('lib/queries/')) { + return 'invalid_lib_prefix'; + } + if (name.startsWith('commands/')) return 'command'; + if (name.startsWith('queries/')) return 'query'; return 'partial'; } /** * Build the expected disk path for a missing platformOS file. - * @param {'partial'|'command'|'query'|'module'} type + * @param {'partial'|'command'|'query'|'module'|'invalid_lib_prefix'} type * @param {string|null} name * @returns {string} */ @@ -538,10 +511,14 @@ function buildCreatePath(type, name) { if (!name) return '(unknown path)'; switch (type) { case 'command': - case 'query': { - // Name may come with or without lib/ prefix — normalize to avoid app/lib/lib/... - const stripped = name.replace(/^lib\//, ''); - return `app/lib/${stripped}.liquid`; + case 'query': + return `app/lib/${name}.liquid`; + case 'invalid_lib_prefix': { + // The path is wrong, not the file. Show where the corrected call + // *would* resolve so the agent can sanity-check that the existing + // file is the intended target before applying the rule's text edit. + const corrected = name.slice('lib/'.length); + return `app/lib/${corrected}.liquid`; } case 'module': { const moduleName = name.split('/')[1] ?? name; @@ -579,3 +556,62 @@ export async function enrichAll(diagnostics, ctx) { return Promise.all(diagnostics.map(d => enrichError(d, { ...ctx, _hoverCache: hoverCache }))); } + +/** + * Bridge rule-engine attribution onto diagnostics that didn't pass through + * `enrichAll` — structural warnings, schema/translation/GraphQL validators, + * diff-aware RemovedRender/AddedParam, new-partial caller check. Those are + * pushed into `result.errors/warnings` AFTER `enrichAll` returns, so their + * rule modules never fire and they land in analytics as `.unmatched`. + * + * This helper runs `runRules` on any diagnostic whose `rule_id` is still + * unset and whose `check` has a registered rule module. On a match it copies + * the rule's `rule_id`, `hint_md`, `confidence`, `see_also`, `fixes`, and + * `case_base_signal` onto the diagnostic — same fields the main enrichAll + * path writes, so downstream (emit loop, fix generator, dashboard) treats + * the diagnostic identically to one that went through enrichAll. + * + * Idempotent: diagnostics already carrying a `rule_id` are skipped so this + * can safely run after enrichAll + structural-warnings push without double + * scoring. + */ +export function bridgeRulesOntoUnattributed(result, ctx) { + const { filePath, content, factGraph, filtersIndex, objectsIndex, tagsIndex, schemaIndex, analyticsStore, projectDir } = ctx; + if (!factGraph) return; + + const facts = { graph: factGraph, filtersIndex, objectsIndex, tagsIndex, schemaIndex, analyticsStore, projectDir }; + + const apply = (d) => { + if (d.rule_id) return; // already attributed + if (!d.check) return; + if (!hasRules(d.check)) return; + + const params = extractParams(d.check, d.message); + const tmplFp = templateOf(d.check, d.message); + const diag = { + check: d.check, + params, + message: d.message, + file: filePath, + line: d.line, + column: d.column ?? 0, + template_fp: tmplFp, + }; + let ruleResult; + try { ruleResult = runRules(diag, facts); } + catch { return; } // runRules failure is non-fatal + if (!ruleResult) return; + + d.rule_id = ruleResult.rule_id; + if (ruleResult.hint_md && !d.hint) d.hint = ruleResult.hint_md; + if (ruleResult.confidence != null && d.confidence == null) d.confidence = ruleResult.confidence; + if (ruleResult.see_also && !d.see_also) d.see_also = ruleResult.see_also; + if (ruleResult.case_base_signal && !d.case_base_signal) d.case_base_signal = ruleResult.case_base_signal; + if (ruleResult.fixes?.length > 0 && !d.fixes) d.fixes = ruleResult.fixes; + attachSeeAlso(d, content); + }; + + for (const d of result.errors) apply(d); + for (const d of result.warnings) apply(d); + for (const d of result.infos) apply(d); +} diff --git a/src/core/fix-generator.js b/src/core/fix-generator.js index aa6c8b4..1f96938 100644 --- a/src/core/fix-generator.js +++ b/src/core/fix-generator.js @@ -11,7 +11,7 @@ import { walk, NodeTypes } from '@platformos/liquid-html-parser'; import { existsSync } from 'node:fs'; import { join } from 'node:path'; -import { extractVarName } from './objects-index.js'; +import { extractParams } from './diagnostic-record.js'; import { isShopifyObject, isShopifyFilter } from './knowledge-loader.js'; import { offsetToLineCol, lineColToOffset, slugFromPath } from './position-utils.js'; import { POSITION_FUZZY_TOLERANCE } from './constants.js'; @@ -271,24 +271,13 @@ function findFilterAt(filterIndex, line, col, filterName) { return null; } -// ── Extract names from diagnostic messages ────────────────────────────────── - -function extractFilterName(message) { - if (!message) return null; - const m = message.match(/`([^`]+)`/) || message.match(/"([^"]+)"/) || message.match(/'([^']+)'/); - return m ? m[1] : null; -} - -function extractPartialPath(message) { - if (!message) return null; - const m = message.match(/['"`]([^'"`]+)['"`]/); - return m ? m[1] : null; -} +// Extraction functions (extractFilterName, extractPartialPath) have been +// centralized into diagnostic-record.js extractParams(). See roadmap §A2. // ── Fix handlers per check type ───────────────────────────────────────────── function fixUndefinedObject(diagnostic, varIndex, isPartialLike, objectsIndex) { - const varName = extractVarName(diagnostic.message); + const varName = extractParams(diagnostic.check, diagnostic.message).variable ?? null; if (!varName) return null; // Shopify object — don't suggest a platformOS replacement (semantic mismatch) @@ -339,7 +328,7 @@ function fixUndefinedObject(diagnostic, varIndex, isPartialLike, objectsIndex) { } function fixUnknownFilter(diagnostic, filterIndex, filtersIndex, tagsIndex) { - const filterName = extractFilterName(diagnostic.message); + const filterName = extractParams(diagnostic.check, diagnostic.message).filter ?? null; if (!filterName) return null; // Tag-as-filter (highest priority) @@ -386,22 +375,9 @@ function fixUnknownFilter(diagnostic, filterIndex, filtersIndex, tagsIndex) { } function fixMissingPartial(diagnostic, projectDir, ast, content) { - const partialPath = extractPartialPath(diagnostic.message); + const partialPath = extractParams(diagnostic.check, diagnostic.message).partial ?? null; if (!partialPath) return null; - // Determine the correct directory based on the partial path - let targetPath; - let fileType = 'partial'; - if (partialPath.startsWith('commands/') || partialPath.startsWith('lib/commands/')) { - targetPath = `app/lib/commands/${partialPath.replace(/^(lib\/)?commands\//, '')}.liquid`; - fileType = 'command'; - } else if (partialPath.startsWith('queries/') || partialPath.startsWith('lib/queries/')) { - targetPath = `app/lib/queries/${partialPath.replace(/^(lib\/)?queries\//, '')}.liquid`; - fileType = 'query'; - } else { - targetPath = `app/views/partials/${partialPath}.liquid`; - } - // Module paths: never create_file — agent cannot create files inside installed modules if (partialPath.startsWith('modules/')) { if (diagnostic.suggestion) { @@ -416,6 +392,36 @@ function fixMissingPartial(diagnostic, projectDir, ast, content) { }; } + // Invalid `lib/` prefix on a function call. `function` tag paths resolve + // from the partial search paths (`app/views/partials/`, `app/lib/`), not + // project root, so `lib/commands/X` expands to `app/lib/lib/commands/X` + // which never exists. Emit a text_edit that strips the prefix; do NOT + // propose creating a phantom file at `app/lib/lib/...`. + if (partialPath.startsWith('lib/commands/') || partialPath.startsWith('lib/queries/')) { + const corrected = partialPath.slice('lib/'.length); + const edit = buildLibPrefixTextEdit(diagnostic, partialPath, corrected, content); + if (edit) return edit; + return { + type: 'guidance', + description: + `Drop the \`lib/\` prefix from \`${partialPath}\`. Function tag paths resolve from ` + + `\`app/lib/\`, so use \`${corrected}\` instead.`, + }; + } + + // Determine the correct directory based on the partial path + let targetPath; + let fileType = 'partial'; + if (partialPath.startsWith('commands/')) { + targetPath = `app/lib/${partialPath}.liquid`; + fileType = 'command'; + } else if (partialPath.startsWith('queries/')) { + targetPath = `app/lib/${partialPath}.liquid`; + fileType = 'query'; + } else { + targetPath = `app/views/partials/${partialPath}.liquid`; + } + // Check if file already exists — if so, don't suggest creating it again if (projectDir) { const absTarget = join(projectDir, targetPath); @@ -438,6 +444,42 @@ function fixMissingPartial(diagnostic, projectDir, ast, content) { }; } +/** + * Build a `text_edit` fix that strips the invalid `lib/` prefix from a + * `function` tag call. Returns null if the diagnostic lacks the position + * fields the edit needs (line/column/endColumn). + * + * Quote handling: when `content` is available, peek at the source byte + * under `diagnostic.column` and re-emit with the same quote style the user + * wrote (`'` or `"`). Otherwise fall back to single-quote, which is the + * convention everywhere in platformOS templates and our scaffolds. + */ +function buildLibPrefixTextEdit(diagnostic, partialPath, corrected, content) { + if (diagnostic.line == null || diagnostic.column == null || diagnostic.endColumn == null) { + return null; + } + let quote = "'"; + if (typeof content === 'string') { + const lines = content.split('\n'); + const sourceLine = lines[diagnostic.line]; + if (typeof sourceLine === 'string' && diagnostic.column < sourceLine.length) { + const ch = sourceLine[diagnostic.column]; + if (ch === "'" || ch === '"') quote = ch; + } + } + return { + type: 'text_edit', + range: { + start: { line: diagnostic.line, character: diagnostic.column }, + end: { line: diagnostic.endLine ?? diagnostic.line, character: diagnostic.endColumn }, + }, + new_text: `${quote}${corrected}${quote}`, + description: + `Drop invalid \`lib/\` prefix — function tag paths resolve from \`app/lib/\`. ` + + `Replace \`${partialPath}\` with \`${corrected}\`.`, + }; +} + /** * Detect if a parameter name likely refers to a collection (array of items). * Used to generate appropriate scaffold content (iteration vs property access). @@ -1118,65 +1160,82 @@ function fixInvalidFrontMatter(diagnostic, content) { } function extractLayoutPath(message) { - const match = message?.match(/`([^`]+)`.*not found/); - if (!match) return 'app/views/layouts/application.html.liquid'; - return `app/views/layouts/${match[1]}.html.liquid`; + // The structural emitter (`validateLayout` in structural-warnings.js) has + // access to projectDir and detects whether the project standardised on + // `.liquid` or `.html.liquid` for layouts. It bakes the full expected + // file path into the message ("Expected file: `app/views/layouts/X.liquid`"), + // so the right thing here is to lift that path verbatim — never re-derive. + const expected = message?.match(/Expected file:\s*`([^`]+)`/); + if (expected) return expected[1]; + + // Defensive fallback: only used when the message shape changes upstream. + // We bias toward `.liquid` (the modern shape) and ignore module layouts — + // an agent shouldn't be creating files inside an installed module anyway. + const layoutName = message?.match(/`([^`]+)`.*not found/)?.[1]; + return layoutName ? `app/views/layouts/${layoutName}.liquid` : 'app/views/layouts/application.liquid'; } +/** + * Heuristic fix for TranslationKeyExists. + * + * Scope (intentionally narrow): produce an actionable text_edit when the + * upstream LSP message contains a "Did you mean 'X'" suggestion AND we + * can locate the offending quoted key on the diagnostic's line. This is + * a complement to the rule-engine `TranslationKeyExists.suggest_nearest` + * — the rule emits guidance, the heuristic emits the diff. + * + * Cases the rule engine OWNS (heuristic must NOT duplicate): + * - `foo[0]` array-index misuse → `TranslationKeyExists.array_index_misuse` + * - generic Levenshtein guidance text → `TranslationKeyExists.suggest_nearest` + * + * Returning null means "no heuristic fix available" — the rule fix (if any) + * stands alone. This is the correct behavior when we can't produce an + * actionable edit. + */ function fixTranslationKeyExists(diagnostic, content) { const msg = diagnostic.message || ''; - // v0.3.3+ includes Levenshtein suggestion: "Did you mean 'correct.key'?" - const suggestMatch = msg.match(/[Dd]id you mean\s+['"`]([^'"`]+)['"`]/); - if (!suggestMatch) { - return { - type: 'guidance', - description: 'Translation key not found. Add it to app/translations/en.yml, or check for typos in the key name.', - }; - } - - const suggestedKey = suggestMatch[1]; - // Extract the wrong key from the message const wrongKeyMatch = msg.match(/['"`]([^'"`]+)['"`]/); const wrongKey = wrongKeyMatch ? wrongKeyMatch[1] : null; - if (!wrongKey || wrongKey === suggestedKey) { - return { - type: 'guidance', - description: `Did you mean \`${suggestedKey}\`? Fix the translation key.`, - }; - } - - // Find the wrong key string at the diagnostic position - const lines = content.split('\n'); - const line = lines[diagnostic.line]; - if (!line) { - return { - type: 'guidance', - description: `Replace translation key \`${wrongKey}\` with \`${suggestedKey}\`.`, - }; - } + // Array-index misuse is owned by the rule engine. Don't emit guidance + // here — it would duplicate (and risk diverging from) the rule's hint. + if (wrongKey && /\[\d+\]/.test(wrongKey)) return null; - // Look for the wrong key as a quoted string in the line - const keyPatterns = [`'${wrongKey}'`, `"${wrongKey}"`]; - for (const pattern of keyPatterns) { - const idx = line.indexOf(pattern); - if (idx >= 0) { - const quote = pattern[0]; - return { - type: 'text_edit', - range: { - start: { line: diagnostic.line, character: idx }, - end: { line: diagnostic.line, character: idx + pattern.length }, - }, - new_text: `${quote}${suggestedKey}${quote}`, - description: `Replace \`${wrongKey}\` with \`${suggestedKey}\``, - }; + // When the LSP message carries a "did you mean 'X'" suggestion AND we can + // locate the quoted key on the line, produce a text_edit. This is the + // ONLY case where the heuristic outranks rule guidance, because text_edits + // are actionable diffs the rule layer cannot produce. + const suggestMatch = msg.match(/[Dd]id you mean\s+['"`]([^'"`]+)['"`]/); + if (suggestMatch && wrongKey && wrongKey !== suggestMatch[1]) { + const suggestedKey = suggestMatch[1]; + const lines = content.split('\n'); + const line = lines[diagnostic.line]; + if (line) { + for (const pattern of [`'${wrongKey}'`, `"${wrongKey}"`]) { + const idx = line.indexOf(pattern); + if (idx >= 0) { + const quote = pattern[0]; + return { + type: 'text_edit', + range: { + start: { line: diagnostic.line, character: idx }, + end: { line: diagnostic.line, character: idx + pattern.length }, + }, + new_text: `${quote}${suggestedKey}${quote}`, + description: `Replace \`${wrongKey}\` with \`${suggestedKey}\``, + }; + } + } } } + // Generic guidance is the safety net for the case where the rule engine + // is not registered / facts are missing. In normal operation the merge + // loop in validate-code.js DROPS this guidance because the rule engine + // produces an attributed equivalent. See the precedence comment there. return { type: 'guidance', - description: `Replace translation key \`${wrongKey}\` with \`${suggestedKey}\`.`, + description: 'Translation key not found. Add it to app/translations/en.yml, or check for typos in the key name.', }; } @@ -1325,10 +1384,18 @@ export function generateFixes(diagnostics, ast, content, filePath, ctx, projectD if (!fix) continue; + // I1 — rule attribution for heuristic fixes. Tagging once here keeps every + // per-check branch above free of boilerplate. The emit loop propagates this + // into the proposed_fixes.rule_id column so Rule Performance can attribute + // adoption to a specific heuristic variant (heuristic:.). + if (!fix.rule_id) { + fix.rule_id = `heuristic:${d.check ?? 'Unknown'}.${fix.type ?? 'fix'}`; + } + if (fix.type === 'add_doc_param') { docParamFixes.push({ index: i, ...fix }); // Attach per-diagnostic fix reference - diagnosticFixes.set(i, { type: 'add_doc_param', description: fix.description, param_name: fix.param_name }); + diagnosticFixes.set(i, { type: 'add_doc_param', description: fix.description, param_name: fix.param_name, rule_id: fix.rule_id }); } else { diagnosticFixes.set(i, fix); // Deduplicate: don't add identical fixes diff --git a/src/core/fs-watcher.js b/src/core/fs-watcher.js index d09a3f3..f1d4453 100644 --- a/src/core/fs-watcher.js +++ b/src/core/fs-watcher.js @@ -283,7 +283,7 @@ async function resyncFile(absPath, lsp, log, emit, counters, hooks = {}) { // Also send workspace/didChangeWatchedFiles — LSPs that honor this spec // method take it as a hint to re-scan. Harmless on LSPs that do not. notifyDidChangeWatched(lsp, uri, 2 /* Changed */, log, counters); - emit('fs_watcher_sync', { path: absPath }); + emit('fs_watcher_sync', { path: absPath, rel_path: relPath }); } } catch (e) { counters.errors++; diff --git a/src/core/intent-validator.js b/src/core/intent-validator.js index adfdaa0..6853a6e 100644 --- a/src/core/intent-validator.js +++ b/src/core/intent-validator.js @@ -809,7 +809,7 @@ export function validatePolicy(changes, projectMap) { // P5: Orphan partial — created but not referenced in plan or project. // Uses the shared predicate so we agree with analyze_project and the - // dep-graph dead-code detector. + // dep-graph orphaned-file detector. if (role === 'partial' && action === 'create') { const name = partialPathToName(path); if (isOrphanPartial(name, projectMap, { planReferencedPartials: referencedPartials })) { diff --git a/src/core/knowledge-loader.js b/src/core/knowledge-loader.js index f5cb867..d432ff8 100644 --- a/src/core/knowledge-loader.js +++ b/src/core/knowledge-loader.js @@ -1,20 +1,24 @@ /** * Knowledge system loader — structured check rules, domain gotchas, Shopify awareness. * - * Loads knowledge.json once at startup and provides lookup functions: + * Loads from split data files (per-check YAML, shopify-objects.json, etc.) with + * fallback to monolithic knowledge.json for backward compatibility. Provides + * lookup functions: * getCheckKnowledge(checkName, context) — hint + Shopify detection for a check * getTriggeredGotchas(domain, triggers) — domain gotchas matching current code state * isShopifyObject(name) — true if name is a Shopify-only object * isShopifyFilter(name) — true if name is a Shopify-only filter */ -import { readFileSync, existsSync } from 'node:fs'; +import { readFileSync, existsSync, readdirSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; +import yaml from 'js-yaml'; const __dirname = dirname(fileURLToPath(import.meta.url)); +const DATA_DIR = join(__dirname, '..', 'data'); -const DEFAULT_LOCATIONS = [ - join(__dirname, '..', 'data', 'knowledge.json'), +const FALLBACK_LOCATIONS = [ + join(DATA_DIR, 'knowledge.json'), join(__dirname, '..', '..', 'data', 'knowledge.json'), ]; @@ -24,7 +28,7 @@ let _shopifyFilters = null; let _shopifyContamination = null; const DEFAULT_SHOPIFY_CONTAMINATION_LOCATIONS = [ - join(__dirname, '..', 'data', 'shopify-contamination.json'), + join(DATA_DIR, 'shopify-contamination.json'), join(__dirname, '..', '..', 'data', 'shopify-contamination.json'), ]; @@ -40,34 +44,67 @@ function loadShopifyContamination() { } } -/** - * Load knowledge.json (lazy, cached). - */ +function loadYaml(filePath) { + if (!existsSync(filePath)) return null; + try { return yaml.load(readFileSync(filePath, 'utf8')); } catch { return null; } +} + +function loadJson(filePath) { + if (!existsSync(filePath)) return null; + try { return JSON.parse(readFileSync(filePath, 'utf8')); } catch { return null; } +} + +function loadFromSplitFiles() { + const kb = { checks: {}, domains: {}, language_features: {}, content_triggers: [], modules_missing_docs: { known: [] } }; + + const checksDir = join(DATA_DIR, 'checks'); + if (existsSync(checksDir)) { + for (const file of readdirSync(checksDir)) { + if (!file.endsWith('.yml') && !file.endsWith('.yaml')) continue; + const check = loadYaml(join(checksDir, file)); + if (check?.name) kb.checks[check.name] = check; + } + } + if (Object.keys(kb.checks).length === 0) return null; + + const shopify = loadJson(join(DATA_DIR, 'shopify-objects.json')); + if (shopify) { + if (kb.checks.UndefinedObject) kb.checks.UndefinedObject.shopify_objects = shopify.objects ?? []; + if (kb.checks.UnknownFilter) kb.checks.UnknownFilter.shopify_filters = shopify.filters ?? []; + } + + kb.domains = loadYaml(join(DATA_DIR, 'domain-gotchas.yml')) ?? {}; + kb.content_triggers = loadYaml(join(DATA_DIR, 'content-triggers.yml')) ?? []; + kb.language_features = loadYaml(join(DATA_DIR, 'language-features.yml')) ?? {}; + kb.modules_missing_docs = loadJson(join(DATA_DIR, 'modules-missing-docs.json')) ?? { known: [] }; + + return kb; +} + function load() { if (_knowledge) return _knowledge; - const path = DEFAULT_LOCATIONS.find(p => existsSync(p)); - if (!path) return null; - try { - _knowledge = JSON.parse(readFileSync(path, 'utf8')); - // Build lookup sets for Shopify awareness - _shopifyObjects = new Set(); - _shopifyFilters = new Set(); - const uo = _knowledge.checks?.UndefinedObject; - if (uo?.shopify_objects) uo.shopify_objects.forEach(o => _shopifyObjects.add(o)); - const uf = _knowledge.checks?.UnknownFilter; - if (uf?.shopify_filters) uf.shopify_filters.forEach(f => _shopifyFilters.add(f)); - return _knowledge; - } catch { - return null; + + _knowledge = loadFromSplitFiles(); + if (!_knowledge) { + const jsonPath = FALLBACK_LOCATIONS.find(p => existsSync(p)); + if (jsonPath) { + try { _knowledge = JSON.parse(readFileSync(jsonPath, 'utf8')); } catch { _knowledge = null; } + } } + if (!_knowledge) return null; + + _shopifyObjects = new Set(); + _shopifyFilters = new Set(); + const uo = _knowledge.checks?.UndefinedObject; + if (uo?.shopify_objects) uo.shopify_objects.forEach(o => _shopifyObjects.add(o)); + const uf = _knowledge.checks?.UnknownFilter; + if (uf?.shopify_filters) uf.shopify_filters.forEach(f => _shopifyFilters.add(f)); + + return _knowledge; } /** * Get structured knowledge for a check type. - * - * @param {string} checkName — e.g., 'UndefinedObject' - * @param {'page'|'partial'|'default'|null} context — file context for context-aware hints - * @returns {{ summary: string, hint: string, shopify_guidance?: string }|null} */ export function getCheckKnowledge(checkName, context = 'default') { const kb = load(); @@ -81,10 +118,6 @@ export function getCheckKnowledge(checkName, context = 'default') { /** * Evaluate triggered gotchas for a domain given the current code state. - * - * @param {string} domain — e.g., 'pages', 'partials' - * @param {object} triggers — { checks: Set, tags: Set, filters: Set } - * @returns {{ rule: string, gotchas: Array<{ id, message, severity }> }} */ export function getTriggeredGotchas(domain, triggers = {}) { const kb = load(); @@ -104,146 +137,78 @@ export function getTriggeredGotchas(domain, triggers = {}) { return { rule: domainDef.rule, gotchas: matched }; } -/** - * Evaluate a trigger condition string. - * - * Supported formats: - * "always" — always true - * "has_check:CheckName" — true if CheckName in checks set - * "uses_tag:tagname" — true if tagname in tags set - * "uses_filter:filtername" — true if filtername in filters set - */ function evaluateTrigger(trigger, checks, tags, filters) { if (!trigger) return false; if (trigger === 'always') return true; - if (trigger.startsWith('has_check:')) { - return checks.has(trigger.slice('has_check:'.length)); - } - if (trigger.startsWith('uses_tag:')) { - return tags.has(trigger.slice('uses_tag:'.length)); - } - if (trigger.startsWith('uses_filter:')) { - return filters.has(trigger.slice('uses_filter:'.length)); - } + if (trigger.startsWith('has_check:')) return checks.has(trigger.slice('has_check:'.length)); + if (trigger.startsWith('uses_tag:')) return tags.has(trigger.slice('uses_tag:'.length)); + if (trigger.startsWith('uses_filter:')) return filters.has(trigger.slice('uses_filter:'.length)); return false; } -/** - * Check if a variable name is a Shopify-only object. - */ export function isShopifyObject(name) { load(); return _shopifyObjects?.has(name) ?? false; } -/** - * Return the set of module partial paths KNOWN to be missing {% doc %} blocks. - * - * Used by diagnostic-pipeline.js:suppressModuleTargetParams to distinguish - * "known offender — silently suppressed" from "new offender — surface in the - * module_doc_missing advisory so the agent can report upstream." Empty set - * when knowledge.json lacks the list entirely. - */ export function getKnownModulesMissingDocs() { const kb = load(); const list = kb?.modules_missing_docs?.known ?? []; return new Set(list); } -/** - * Check if a filter name is a Shopify-only filter. - */ export function isShopifyFilter(name) { load(); return _shopifyFilters?.has(name) ?? false; } -/** - * Get rich replacement/note data for a Shopify object name. - * Returns { replacement, note } or null if not in the contamination map. - */ export function getShopifyObject(name) { const data = loadShopifyContamination(); return data?.objects?.[name] ?? null; } -/** - * Get rich replacement/note data for a Shopify filter name. - * Returns { replacement, note } or null if not in the contamination map. - */ export function getShopifyFilter(name) { const data = loadShopifyContamination(); return data?.filters?.[name] ?? null; } -/** - * Get rich replacement/note data for a Shopify tag name. - * Returns { replacement, note } or null if not in the contamination map. - */ export function getShopifyTag(name) { const data = loadShopifyContamination(); return data?.tags?.[name] ?? null; } -/** - * Check if a tag name is a Shopify-only tag (not valid in platformOS). - */ export function isShopifyTag(name) { const data = loadShopifyContamination(); return !!(data?.tags?.[name]); } -/** - * Get the domain rule one-liner. - */ export function getDomainRule(domain) { const kb = load(); return kb?.domains?.[domain]?.rule ?? null; } -/** - * Get a language feature entry (try_catch, theme_render_rc, liquid_doc, etc.). - */ export function getLanguageFeature(featureId) { const kb = load(); return kb?.language_features?.[featureId] ?? null; } -/** - * Evaluate content-based triggers against file content and domain. - * Returns proactive knowledge tips for patterns detected in the code. - * - * @param {string} content — file content to scan - * @param {string} domain — domain from path detection (pages, partials, etc.) - * @returns {Array<{ id: string, message: string, severity: string }>} - */ export function getContentTriggers(content, domain) { const kb = load(); if (!kb?.content_triggers || !content || !domain) return []; const matched = []; for (const trigger of kb.content_triggers) { - // Skip if this trigger doesn't apply to the current domain if (trigger.domains && !trigger.domains.includes(domain)) continue; - try { - const re = new RegExp(trigger.pattern); - if (re.test(content)) { - matched.push({ - id: trigger.id, - message: trigger.message, - severity: trigger.severity, - }); + if (new RegExp(trigger.pattern).test(content)) { + matched.push({ id: trigger.id, message: trigger.message, severity: trigger.severity }); } - } catch { - // Invalid regex in knowledge.json — skip silently - } + } catch {} } return matched; } -/** Reset cache (for testing). */ export function _resetKnowledge() { _knowledge = null; _shopifyObjects = null; diff --git a/src/core/liquid-parser.js b/src/core/liquid-parser.js index e249a4f..7feb677 100644 --- a/src/core/liquid-parser.js +++ b/src/core/liquid-parser.js @@ -26,6 +26,7 @@ export function extractAllFromAST(ast) { let method = null; const seenRenders = new Set(); const renders = []; + const renderCalls = []; const seenGQL = new Set(); const graphql = []; const filters = new Set(); @@ -53,31 +54,57 @@ export function extractAllFromAST(ast) { case NodeTypes.LiquidTag: { tags.add(node.name); if (node.name === NamedTags.render || node.name === 'include') { - // Track both render and include — include is deprecated but still used - // by module APIs (modules/*/helpers/*) for scope sharing. if (typeof node.markup === 'string') { const partialMatch = node.markup.match(/^["']([^"']+)['"]/); - if (partialMatch && !seenRenders.has(partialMatch[1])) { - seenRenders.add(partialMatch[1]); - renders.push(partialMatch[1]); + if (partialMatch) { + const partialName = partialMatch[1]; + if (!seenRenders.has(partialName)) { + seenRenders.add(partialName); + renders.push(partialName); + } + const args = extractArgsFromMarkupString(node.markup); + renderCalls.push({ partial: partialName, args }); } for (const km of node.markup.matchAll(/["']([^"']+)['"]\s*\|\s*t\b/g)) { transKeys.add(km[1]); } } else { const partial = node.markup?.partial; - if (partial?.type === NodeTypes.String && !seenRenders.has(partial.value)) { - seenRenders.add(partial.value); - renders.push(partial.value); + if (partial?.type === NodeTypes.String) { + if (!seenRenders.has(partial.value)) { + seenRenders.add(partial.value); + renders.push(partial.value); + } + const args = extractArgsFromMarkup(node.markup); + renderCalls.push({ partial: partial.value, args }); } } } else if (node.name === NamedTags.graphql) { const markup = node.markup; if (markup?.type === NodeTypes.GraphQLMarkup) { const gqlPath = markup.graphql; - if (gqlPath?.type === NodeTypes.String && !seenGQL.has(gqlPath.value)) { - seenGQL.add(gqlPath.value); - graphql.push({ variable: markup.name, queryName: gqlPath.value }); + if (gqlPath?.type === NodeTypes.String) { + const queryName = gqlPath.value; + const sourceKind = classifyGraphqlSourceKind(node); + const args = extractArgsFromMarkup(markup); + if (seenGQL.has(queryName)) { + // Same op called twice. Keep the first entry but upgrade + // source_kind to the most pessimistic value across calls so + // downstream rules can detect truncation regardless of which + // call won the dedup. + if (sourceKind === 'liquid_multiline_truncated') { + const existing = graphql.find(g => g.queryName === queryName); + if (existing) existing.source_kind = 'liquid_multiline_truncated'; + } + } else { + seenGQL.add(queryName); + graphql.push({ + variable: markup.name, + queryName, + args, + source_kind: sourceKind, + }); + } } } } @@ -116,7 +143,59 @@ export function extractAllFromAST(ast) { } }); - return { slug, layout, method, renders, graphql, filters, tags, transKeys, prompts, docParams }; + return { slug, layout, method, renders, renderCalls, graphql, filters, tags, transKeys, prompts, docParams }; +} + +function extractArgsFromMarkup(markup) { + if (!markup?.args) return []; + return markup.args + .filter(a => a.type === NodeTypes.NamedArgument && typeof a.name === 'string') + .map(a => a.name); +} + +/** + * Classify the surface form of a `{% graphql %}` call. + * + * 'tag' — `{% graphql ... %}` (with delimiters). + * 'liquid_inline' — inside a `{% liquid %}` block, single-line. + * 'liquid_multiline_truncated' — inside a `{% liquid %}` block, written + * with a comma + newline continuation. The + * liquid-html-parser truncates the call at + * the first newline-comma, so `markup.args` + * silently drops every argument past it — + * and pos-cli's LSP diagnostic check has + * the same blind spot. The agent sees the + * args in source; both parsers don't. + * + * Detection criterion for the truncated form: source range starts without + * `{%` (we are inside a `{% liquid %}` block), the visible source text ends + * on a comma, AND the immediately trailing characters in the file contain + * another `name:` clause on a subsequent line. The trailing-text check is + * the load-bearing signal — without it a legitimate inline call that just + * happens to end on a comma (rare, but possible) would be misclassified. + */ +export function classifyGraphqlSourceKind(node) { + const src = typeof node?.source === 'string' ? node.source : ''; + const start = node?.position?.start ?? 0; + const end = node?.position?.end ?? 0; + const text = src.slice(start, end); + if (text.startsWith('{%')) return 'tag'; + if (text.trimEnd().endsWith(',')) { + const trail = src.slice(end, end + 200); + if (/\n\s*[A-Za-z_]\w*\s*:/.test(trail)) { + return 'liquid_multiline_truncated'; + } + } + return 'liquid_inline'; +} + +function extractArgsFromMarkupString(markupStr) { + const args = []; + const afterPartial = markupStr.replace(/^["'][^"']+["']\s*,?\s*/, ''); + for (const m of afterPartial.matchAll(/(\w+)\s*:/g)) { + args.push(m[1]); + } + return args; } /** diff --git a/src/core/lsp-client.js b/src/core/lsp-client.js index 0bf75e5..9c3cdd8 100644 --- a/src/core/lsp-client.js +++ b/src/core/lsp-client.js @@ -125,17 +125,11 @@ export class PlatformOSLSPClient { if (msg.method === 'textDocument/publishDiagnostics') { const uri = msg.params.uri; const diags = msg.params.diagnostics ?? []; - const waiter = this.#diagWaiters.get(uri); - if (waiter?.gate && !waiter.gate()) { - // Pre-barrier notification — stale, discard entirely - return; - } this.#diagnostics.set(uri, diags); + const waiter = this.#diagWaiters.get(uri); if (waiter?.onDiag) { - // Settle-based waiter: notify but keep alive for updates waiter.onDiag(diags); } else if (waiter) { - // Simple waiter: resolve immediately this.#diagWaiters.delete(uri); clearTimeout(waiter.timer); waiter.resolve(diags); @@ -257,48 +251,28 @@ export class PlatformOSLSPClient { * Uses a barrier request to guarantee freshness. The LSP processes stdin * messages sequentially, so after sending: * 1. didOpen/didChange (our content) - * 2. hover request (barrier) - * the LSP must process (1) before responding to (2). Any publishDiagnostics - * arriving in stdout BEFORE the hover response is from a prior analysis - * (stale); anything AFTER is from our content (fresh). - * - * The barrier's resolve callback runs synchronously during #drain (same - * synchronous loop as message processing), so the gate flag is set before - * any subsequent publishDiagnostics in the same buffer chunk is handled. - */ - /** * Sync document content and wait for fresh diagnostics. * - * Uses a two-layer strategy to guarantee freshness: - * - * **Layer 1 — Barrier (hover fence):** The LSP processes stdin messages - * sequentially, so after sending didOpen/didChange + hover, the hover - * response proves the LSP received our content. Any publishDiagnostics - * arriving BEFORE the hover response is from a prior analysis (stale) - * and is discarded by the gate. - * - * **Layer 2 — Settle window:** The LSP may use async background workers - * for analysis. A stale analysis that was already in-flight can publish - * diagnostics AFTER the barrier. The settle window (200ms) ensures we - * accept the LAST publishDiagnostics within a quiet period, not just - * the first. If the LSP sends stale-then-fresh in quick succession, - * the settle timer resets on each arrival and we resolve with the - * latest (fresh) set. + * Sends a hover request as a synchronization fence — the LSP must process + * didOpen/didChange before responding to hover, proving it received our + * content. A settle window (500ms quiet period) then waits for the LAST + * publishDiagnostics batch, accepting all arrivals regardless of barrier + * timing. The LSP may emit valid diagnostics before the hover response + * when analysis is fast, so no pre-barrier filtering is applied. */ awaitDiagnostics(uri, text, timeoutMs = LSP_DIAGNOSTICS_TIMEOUT_MS) { this.syncDoc(uri, text); this.#diagnostics.delete(uri); - // ── Barrier: hover request used as a synchronization fence ── - let barrierPassed = false; + // ── Barrier: hover request as sync fence (ensures LSP processes our content) ── const barrierId = ++this.#reqId; const barrierTimer = setTimeout(() => { - if (this.#pending.delete(barrierId)) barrierPassed = true; + this.#pending.delete(barrierId); }, Math.min(timeoutMs, LSP_BARRIER_TIMEOUT_MS)); this.#pending.set(barrierId, { - resolve: () => { clearTimeout(barrierTimer); barrierPassed = true; }, - reject: () => { clearTimeout(barrierTimer); barrierPassed = true; }, + resolve: () => { clearTimeout(barrierTimer); }, + reject: () => { clearTimeout(barrierTimer); }, }); this.#send({ jsonrpc: '2.0', id: barrierId, @@ -306,7 +280,7 @@ export class PlatformOSLSPClient { params: { textDocument: { uri }, position: { line: 0, character: 0 } }, }); - // ── Diagnostic waiter: barrier gate + settle window ── + // ── Diagnostic waiter: settle window resolves with latest batch ── const SETTLE_MS = DIAGNOSTICS_SETTLE_MS; return new Promise((resolve) => { let latestDiags = null; @@ -328,15 +302,13 @@ export class PlatformOSLSPClient { this.#diagWaiters.set(uri, { timer: mainTimer, settleTimer: null, - gate: () => barrierPassed, onDiag: (diags) => { latestDiags = diags; if (settleTimer) clearTimeout(settleTimer); settleTimer = setTimeout(() => finish(latestDiags), SETTLE_MS); - // Store ref for crash cleanup this.#diagWaiters.get(uri).settleTimer = settleTimer; }, - resolve: (diags) => finish(diags), // crash cleanup path + resolve: (diags) => finish(diags), }); }); } diff --git a/src/core/module-scanner.js b/src/core/module-scanner.js index 87872ce..127a3ef 100644 --- a/src/core/module-scanner.js +++ b/src/core/module-scanner.js @@ -40,6 +40,8 @@ export async function scanModule(projectDir, moduleName) { display_name: metadata.name || moduleName, version: metadata.version || 'unknown', dependencies: metadata.dependencies || {}, + manifest_source: metadata.manifest_source ?? null, + ...(metadata.manifest_warnings ? { manifest_warnings: metadata.manifest_warnings } : {}), installed: true, ...apiSurface, schemas, @@ -71,6 +73,8 @@ export async function listModules(projectDir) { display_name: meta.name || entry.name, version: meta.version || 'unknown', dependencies: meta.dependencies || {}, + manifest_source: meta.manifest_source ?? null, + ...(meta.manifest_warnings ? { manifest_warnings: meta.manifest_warnings } : {}), }); } @@ -81,28 +85,107 @@ export async function listModules(projectDir) { } // ── Metadata scanning ──────────────────────────────────────────────────────── +// +// Precedence (most authoritative first): +// 1. `pos-module.json` — upstream platformOS module manifest. Source +// of truth for `version` and `dependencies`. +// 2. `template-values.json` — generated artifact emitted by +// `pos-cli modules version`. Mirrors +// pos-module.json but can drift if deps are +// added without re-running the version sync. +// 3. `package.json` — npm metadata. Its `version` reflects the +// npm-package layout, NOT the platformOS +// module version. Last-resort fallback. +// +// When both `pos-module.json` and `template-values.json` exist we run a drift +// check; any divergence in `version` or in the `dependencies` key set surfaces +// in `manifest_warnings` so module_info can flag it for the operator. + +async function readJsonOr(file) { + try { + return JSON.parse(await readFile(file, 'utf8')); + } catch { + return null; + } +} async function scanMetadata(moduleDir) { - // Try template-values.json first (platformOS module metadata) - const tvPath = join(moduleDir, 'template-values.json'); - try { - const content = await readFile(tvPath, 'utf8'); - return JSON.parse(content); - } catch {} + const posModule = await readJsonOr(join(moduleDir, 'pos-module.json')); + const templateValues = await readJsonOr(join(moduleDir, 'template-values.json')); + + let primary = null; + let source = null; + if (posModule) { + primary = posModule; + source = 'pos-module.json'; + } else if (templateValues) { + primary = templateValues; + source = 'template-values.json'; + } else { + const pkg = await readJsonOr(join(moduleDir, 'package.json')); + if (pkg) { + return { + name: pkg.name ?? null, + version: pkg.version ?? null, + dependencies: pkg.dependencies ?? {}, + manifest_source: 'package.json', + }; + } + return { manifest_source: null }; + } - // Fallback to package.json - const pkgPath = join(moduleDir, 'package.json'); - try { - const content = await readFile(pkgPath, 'utf8'); - const pkg = JSON.parse(content); - return { - name: pkg.name, - version: pkg.version, - dependencies: pkg.dependencies || {}, - }; - } catch {} + const out = { + name: primary.name ?? null, + version: primary.version ?? null, + dependencies: primary.dependencies ?? {}, + manifest_source: source, + }; + + if (posModule && templateValues) { + const warnings = detectManifestDrift(posModule, templateValues); + if (warnings.length > 0) out.manifest_warnings = warnings; + } + + return out; +} + +/** + * Compare `pos-module.json` and `template-values.json` and emit one warning per + * detected divergence. Used by scanMetadata to surface stale module-version + * sync state — the canonical fix is to re-run `pos-cli modules version `. + */ +function detectManifestDrift(posModule, templateValues) { + const warnings = []; + + if ((posModule.version ?? null) !== (templateValues.version ?? null)) { + warnings.push({ + kind: 'version_drift', + pos_module: posModule.version ?? null, + template_values: templateValues.version ?? null, + message: `pos-module.json (${posModule.version ?? 'null'}) and template-values.json (${templateValues.version ?? 'null'}) report different versions. pos-module.json wins. Re-run \`pos-cli modules version \` to sync.`, + }); + } + + const posDeps = posModule.dependencies ?? {}; + const tvDeps = templateValues.dependencies ?? {}; + const posKeys = Object.keys(posDeps); + const tvKeys = Object.keys(tvDeps); + const onlyPos = posKeys.filter(k => !(k in tvDeps)); + const onlyTv = tvKeys.filter(k => !(k in posDeps)); + + if (onlyPos.length > 0 || onlyTv.length > 0) { + warnings.push({ + kind: 'dependency_drift', + only_in_pos_module: onlyPos.sort(), + only_in_template_values: onlyTv.sort(), + message: [ + onlyPos.length > 0 ? `pos-module.json adds [${onlyPos.sort().join(', ')}]` : null, + onlyTv.length > 0 ? `template-values.json adds [${onlyTv.sort().join(', ')}]` : null, + ].filter(Boolean).join('; ') + '. pos-module.json wins. Re-run `pos-cli modules version ` to sync.', + }); + } - return {}; + return warnings; } // ── Public API surface ─────────────────────────────────────────────────────── diff --git a/src/core/orphan-detector.js b/src/core/orphan-detector.js index cab64a8..c97287c 100644 --- a/src/core/orphan-detector.js +++ b/src/core/orphan-detector.js @@ -6,7 +6,7 @@ * * 1. `analyze-project.js` checkIntegrity → iterated `projectMap.partials` * and emitted `orphan_partial` issues whenever `rendered_by.length === 0`. - * 2. `dependency-graph.js` `detectDeadCode` → iterated the graph and + * 2. `dependency-graph.js` `detectOrphanedFiles` → iterated the graph and * applied entry-point exemptions (pages, subphases, graphql). * 3. `intent-validator.js` P5 → checked one partial at a time, combining * project-state evidence with plan-local `referencedPartials`. @@ -39,7 +39,7 @@ export function isPartialRendered(partialName, projectMap) { * Use for: * - validate_intent P5 (new partial about to be created) * - analyze_project integrity (existing partials without renderers) - * - dependency-graph dead-code detection (partial-class files) + * - dependency-graph orphaned-file detection (partial-class files) * * A partial is NOT orphaned when any of the following is true: * - any file in project_map renders it @@ -86,7 +86,7 @@ export function findOrphanPartials(projectMap, { planReferencedPartials } = {}) * Subphase command/query files live under /build/, /check/, /execute/ and are * called by convention at runtime — parent command files rarely wire them * explicitly, so static analysis always marks them orphaned. They must never - * be flagged as dead code. + * be flagged as orphaned. */ export function isSubphaseFile(path) { return /\/build(\/|\.liquid$)|\/check(\/|\.liquid$)|\/execute(\/|\.liquid$)/.test(path); diff --git a/src/core/page-route-index.js b/src/core/page-route-index.js index 42aae1f..b1f661f 100644 --- a/src/core/page-route-index.js +++ b/src/core/page-route-index.js @@ -27,15 +27,38 @@ */ import { existsSync, readdirSync, readFileSync } from 'node:fs'; -import { join, relative, sep } from 'node:path'; +import { isAbsolute, join, relative, sep } from 'node:path'; const PAGES_SUBDIR = 'app/views/pages'; /** + * Build a route → methods index from the project's pages directory. + * + * The optional `overlay` represents the file CURRENTLY under validation. Its + * in-memory content is used in place of the on-disk version (or in addition + * to, if the file does not yet exist on disk). This is load-bearing for the + * self-page case: when the agent runs validate_code on `app/views/pages/index.liquid` + * with `method: post` in-memory frontmatter while disk still has no method + * declaration, the LSP fires `MissingPage` for route `/` (POST). The on-disk + * scan alone would not see the in-memory frontmatter and the false positive + * would survive verification — even though, the moment the agent writes, + * the route IS served. + * + * Overlay rules: + * - filePath may be relative (resolved against projectDir) or absolute. + * - Only files under `app/views/pages/` are considered. A non-page overlay + * (e.g., a partial under validation) is ignored — the index covers pages + * only. + * - The overlay is treated as the source of truth for that one file. When + * the same path also exists on disk, the disk read is skipped and the + * overlay frontmatter wins. When the path does not exist on disk yet, + * the overlay adds a new entry to the index. + * * @param {string} projectDir + * @param {{ filePath: string, content: string } | null} [overlay] * @returns {{ routes: Map> }} */ -export function buildPageRouteIndex(projectDir) { +export function buildPageRouteIndex(projectDir, overlay = null) { const routes = new Map(); if (!projectDir) return { routes }; @@ -58,10 +81,31 @@ export function buildPageRouteIndex(projectDir) { } } + // Resolve the overlay (if any) to an absolute path under the pages root. + // Overlays outside `app/views/pages/` are ignored — the route index covers + // pages only, and partials/layouts/assets cannot serve a route. + let overlayAbs = null; + if ( + overlay && + typeof overlay.filePath === 'string' && + typeof overlay.content === 'string' && + /\.liquid$/.test(overlay.filePath) + ) { + const abs = isAbsolute(overlay.filePath) ? overlay.filePath : join(projectDir, overlay.filePath); + if (abs === rootAbs || abs.startsWith(rootAbs + sep)) { + overlayAbs = abs; + if (!files.includes(abs)) files.push(abs); + } + } + for (const abs of files) { let raw; - try { raw = readFileSync(abs, 'utf8'); } - catch { continue; } + if (abs === overlayAbs) { + raw = overlay.content; + } else { + try { raw = readFileSync(abs, 'utf8'); } + catch { continue; } + } const rel = relative(rootAbs, abs).split(sep).join('/'); const { slug, method } = extractFrontmatter(raw); diff --git a/src/core/project-fact-graph.js b/src/core/project-fact-graph.js new file mode 100644 index 0000000..2a27e9b --- /dev/null +++ b/src/core/project-fact-graph.js @@ -0,0 +1,285 @@ +import { resolveRenderTarget, resolveFunctionTarget, resolveGraphqlTarget } from './dependency-graph.js'; + +export function buildFactGraph(projectMap) { + return new ProjectFactGraph(projectMap); +} + +class ProjectFactGraph { + constructor(projectMap) { + this._map = projectMap; + this._nodes = new Map(); + this._byType = new Map(); + this._dependsOn = new Map(); + this._referencedBy = new Map(); + this._renderCalls = new Map(); + + this._indexNodes(); + this._buildEdges(); + this._indexRenderCalls(); + } + + _addNode(type, key, path, props = {}) { + const node = Object.freeze({ type, key, path, ...props }); + if (path) this._nodes.set(path, node); + if (!this._byType.has(type)) this._byType.set(type, new Map()); + this._byType.get(type).set(key, node); + return node; + } + + _addEdge(source, target) { + if (!source || !target) return; + if (!this._dependsOn.has(source)) this._dependsOn.set(source, new Set()); + if (!this._referencedBy.has(target)) this._referencedBy.set(target, new Set()); + this._dependsOn.get(source).add(target); + this._referencedBy.get(target).add(source); + } + + _indexNodes() { + const m = this._map; + + for (const [key, page] of Object.entries(m.pages ?? {})) { + this._addNode('page', key, page.path, { + slug: page.slug, method: page.method, layout: page.layout, + renders: page.renders, render_calls: page.render_calls, + function_calls: page.function_calls, + graphql_calls: page.graphql_calls, + }); + } + + for (const [name, partial] of Object.entries(m.partials ?? {})) { + this._addNode('partial', name, partial.path, { + params: partial.params, renders: partial.renders, + render_calls: partial.render_calls, + function_calls: partial.function_calls, rendered_by: partial.rendered_by, + graphql_calls: partial.graphql_calls, + }); + } + + for (const [path, cmd] of Object.entries(m.commands ?? {})) { + this._addNode('command', path, path, { + params: cmd.params, phases: cmd.phases, + graphql_calls: cmd.graphql_calls, function_calls: cmd.function_calls, + }); + } + + for (const [path, q] of Object.entries(m.queries ?? {})) { + this._addNode('query', path, path, { + params: q.params, graphql_calls: q.graphql_calls, + function_calls: q.function_calls, + }); + } + + for (const [name, gql] of Object.entries(m.graphql ?? {})) { + this._addNode('graphql', name, `app/graphql/${name}.graphql`, { + operation: gql.operation, gqlName: gql.name, args: gql.args, table: gql.table, + }); + } + + for (const [name, schema] of Object.entries(m.schema ?? {})) { + this._addNode('schema', name, schema.path, { properties: schema.properties }); + } + + for (const [path, layout] of Object.entries(m.layouts ?? {})) { + this._addNode('layout', path, layout.path, { + renders: layout.renders, + render_calls: layout.render_calls, + function_calls: layout.function_calls, + graphql_calls: layout.graphql_calls, + }); + } + + for (const [locale, keys] of Object.entries(m.translations ?? {})) { + for (const key of Object.keys(keys)) { + this._addNode('translation', `${locale}:${key}`, null, { + locale, key, value: keys[key], + }); + } + } + + for (const asset of (m.assets ?? [])) { + this._addNode('asset', asset, `app/assets/${asset}`); + } + } + + _buildEdges() { + const m = this._map; + + const layoutsByName = {}; + for (const layout of Object.values(m.layouts ?? {})) { + if (!layout?.path) continue; + const name = layout.path + .replace(/^app\/views\/layouts\//, '') + .replace(/\.html\.liquid$/, '') + .replace(/\.liquid$/, ''); + layoutsByName[name] = layout.path; + } + + for (const page of Object.values(m.pages ?? {})) { + if (!page?.path) continue; + for (const r of page.renders ?? []) { + this._addEdge(page.path, resolveRenderTarget(r, m, page.path)); + } + for (const fc of page.function_calls ?? []) { + this._addEdge(page.path, resolveFunctionTarget(fc.path)); + } + if (page.layout) { + const layoutPath = layoutsByName[page.layout]; + if (layoutPath) this._addEdge(page.path, layoutPath); + } + } + + for (const layout of Object.values(m.layouts ?? {})) { + if (!layout?.path) continue; + for (const r of layout.renders ?? []) { + this._addEdge(layout.path, resolveRenderTarget(r, m, layout.path)); + } + for (const fc of layout.function_calls ?? []) { + this._addEdge(layout.path, resolveFunctionTarget(fc.path)); + } + } + + for (const partial of Object.values(m.partials ?? {})) { + if (!partial?.path) continue; + for (const r of partial.renders ?? []) { + this._addEdge(partial.path, resolveRenderTarget(r, m, partial.path)); + } + for (const fc of partial.function_calls ?? []) { + this._addEdge(partial.path, resolveFunctionTarget(fc.path)); + } + } + + for (const [cmdPath, cmd] of Object.entries(m.commands ?? {})) { + for (const fc of cmd.function_calls ?? []) { + this._addEdge(cmdPath, resolveFunctionTarget(fc.path)); + } + for (const g of cmd.graphql_calls ?? []) { + const opName = typeof g === 'string' ? g : g?.queryName; + this._addEdge(cmdPath, resolveGraphqlTarget(opName)); + } + } + + for (const [qPath, q] of Object.entries(m.queries ?? {})) { + for (const fc of q.function_calls ?? []) { + this._addEdge(qPath, resolveFunctionTarget(fc.path)); + } + for (const g of q.graphql_calls ?? []) { + const opName = typeof g === 'string' ? g : g?.queryName; + this._addEdge(qPath, resolveGraphqlTarget(opName)); + } + } + } + + _indexRenderCalls() { + for (const [path, node] of this._nodes) { + const calls = node.render_calls; + if (calls && calls.length > 0) { + this._renderCalls.set(path, calls); + } + } + } + + // ── Query API ── + + nodeByPath(path) { + return this._nodes.get(path) ?? null; + } + + nodesByType(type) { + const m = this._byType.get(type); + return m ? [...m.values()] : []; + } + + nodeByKey(type, key) { + return this._byType.get(type)?.get(key) ?? null; + } + + hasNode(path) { + return this._nodes.has(path); + } + + dependsOn(path) { + return [...(this._dependsOn.get(path) ?? [])]; + } + + referencedBy(path) { + return [...(this._referencedBy.get(path) ?? [])]; + } + + allFiles() { + return [...this._nodes.keys()].sort(); + } + + allLiquidFiles() { + return this.allFiles().filter(f => f.endsWith('.liquid')); + } + + allCheckableFiles() { + return this.allFiles().filter(f => f.endsWith('.liquid') || f.endsWith('.graphql')); + } + + get size() { + return this._nodes.size; + } + + get nodeCount() { + const counts = {}; + for (const [type, m] of this._byType) counts[type] = m.size; + return counts; + } + + get edgeCount() { + let n = 0; + for (const set of this._dependsOn.values()) n += set.size; + return n; + } + + toDependencyGraph() { + const graph = {}; + const seed = (p) => { if (p && !graph[p]) graph[p] = { depends_on: [], referenced_by: [] }; }; + + for (const path of this._nodes.keys()) seed(path); + for (const [source, targets] of this._dependsOn) { + seed(source); + for (const target of targets) { + seed(target); + graph[source].depends_on.push(target); + graph[target].referenced_by.push(source); + } + } + return graph; + } + + checkEdgeIntegrity() { + const missing = []; + for (const [source, targets] of this._dependsOn) { + for (const target of targets) { + if (!this._nodes.has(target)) { + missing.push({ source, target }); + } + } + } + return missing; + } + + renderCallsFrom(filePath) { + return this._renderCalls.get(filePath) ?? []; + } + + renderCallsTo(partialKey) { + const results = []; + for (const [callerPath, calls] of this._renderCalls) { + for (const call of calls) { + if (call.partial === partialKey) { + results.push({ callerPath, args: call.args }); + } + } + } + return results; + } + + partialSignature(partialKey) { + const node = this.nodeByKey('partial', partialKey); + if (!node) return null; + return node.params ?? []; + } +} diff --git a/src/core/project-scanner.js b/src/core/project-scanner.js index 0aa310a..bd0d968 100644 --- a/src/core/project-scanner.js +++ b/src/core/project-scanner.js @@ -53,7 +53,13 @@ export async function scanProject(projectDir) { method, layout: file.structural.layout, renders: file.structural.renders, + render_calls: file.structural.renderCalls, function_calls: file.functionCalls, + // API pages (`method: post`, `format: json`) commonly carry a + // `{% graphql %}` body directly. Indexing graphql_calls here + // lets rule-engine consumers resolve the operation from any + // caller — same shape as commands/queries. + graphql_calls: file.structural.graphql, }; break; } @@ -63,7 +69,11 @@ export async function scanProject(projectDir) { path: file.relPath, params: [...file.structural.docParams], renders: file.structural.renders, + render_calls: file.structural.renderCalls, function_calls: file.functionCalls, + // Partials may host `{% graphql %}` calls (data-fetching helpers). + // Index for the same reason as pages above. + graphql_calls: file.structural.graphql, rendered_by: [], }; break; @@ -88,7 +98,15 @@ export async function scanProject(projectDir) { break; } case 'layouts': { - layouts[file.relPath] = { path: file.relPath }; + layouts[file.relPath] = { + path: file.relPath, + renders: file.structural.renders, + render_calls: file.structural.renderCalls, + function_calls: file.functionCalls, + // Same rationale as pages/partials — layouts may invoke graphql + // for nav data, etc. + graphql_calls: file.structural.graphql, + }; break; } } @@ -111,6 +129,7 @@ export async function scanProject(projectDir) { queries, pages, partials, + layouts, translations, assets, summary: { @@ -283,6 +302,7 @@ async function scanLiquidFiles(appDir) { layout: extracted.layout, method: extracted.method, renders: extracted.renders, + renderCalls: extracted.renderCalls, graphql: extracted.graphql, filters: extracted.filters, tags: extracted.tags, diff --git a/src/core/render-flow.js b/src/core/render-flow.js new file mode 100644 index 0000000..8ba1702 --- /dev/null +++ b/src/core/render-flow.js @@ -0,0 +1,96 @@ +/** + * Render flow analyzer — cross-file variable tracking through render chains. + * + * Pure query functions over ProjectFactGraph. No side effects. + * + * Used by: + * - UnusedAssign rules: suppress when variable is passed to a render call + * - MissingRenderPartialArguments rules: show full signatures, detect chain satisfaction + */ + +/** + * Check if a variable is passed as an argument value in any render call within a file. + * Handles both `{% render 'partial', arg: variable %}` (arg name matches variable) + * and variable references in render call argument values. + */ +export function isVariablePassedToRender(graph, filePath, varName) { + const calls = graph.renderCallsFrom(filePath); + for (const call of calls) { + if (call.args.includes(varName)) return true; + } + return false; +} + +/** + * Check if a variable is passed to any function call in the file's content. + * Function calls use `{% function result = 'path', arg: value %}` — the arg + * names reference variables that are "used". + */ +export function isVariablePassedToFunction(graph, filePath, varName) { + const node = graph.nodeByPath(filePath); + if (!node?.function_calls) return false; + for (const fc of node.function_calls) { + if (fc.variable === varName) return true; + } + return false; +} + +/** + * Get all callers of a partial with the arguments they pass. + */ +export function callersWithArgs(graph, partialKey) { + return graph.renderCallsTo(partialKey); +} + +/** + * Get the declared parameters of a partial from its {% doc %} block. + */ +export function getPartialParams(graph, partialKey) { + return graph.partialSignature(partialKey) ?? []; +} + +/** + * Find which declared parameters a specific caller does NOT pass. + */ +export function missingArgsForCaller(graph, callerPath, partialKey) { + const params = getPartialParams(graph, partialKey); + if (params.length === 0) return []; + + const calls = graph.renderCallsFrom(callerPath); + const call = calls.find(c => c.partial === partialKey); + if (!call) return [...params]; + + const passed = new Set(call.args); + return params.filter(p => !passed.has(p)); +} + +/** + * Check if a missing param on a callee is satisfied by the caller having + * that param in its own signature (received from a grandparent). + * In a chain Page → A → B, if B requires `x` and A doesn't pass it, + * but A declares `x` as a param, A has it in scope and can forward it. + */ +export function isParamAvailableInCallerScope(graph, callerPath, paramName) { + const callerNode = graph.nodeByPath(callerPath); + if (!callerNode?.params) return false; + return callerNode.params.includes(paramName); +} + +/** + * Build a complete render flow summary for a file: all outgoing render calls + * with their args, the target partial's declared params, and missing args. + */ +export function renderFlowSummary(graph, filePath) { + const calls = graph.renderCallsFrom(filePath); + return calls.map(call => { + const targetParams = getPartialParams(graph, call.partial); + const passed = new Set(call.args); + const missing = targetParams.filter(p => !passed.has(p)); + return { + partial: call.partial, + passed_args: call.args, + declared_params: targetParams, + missing_args: missing, + }; + }); +} diff --git a/src/core/rule-overrides.js b/src/core/rule-overrides.js new file mode 100644 index 0000000..f6f65ad --- /dev/null +++ b/src/core/rule-overrides.js @@ -0,0 +1,126 @@ +/** + * Rule overrides — manual force-enable / force-disable records that survive + * restart. Persisted at `/.pos-supervisor/rule-overrides.json`. + * + * Two kinds of override: + * - force_enable: rule runs even when case-base scoring would disable it. + * Use case: operator wants to re-test a rule after fixing + * a false-positive source, before enough fresh outcomes + * accumulate to auto-flip the score. + * - force_disable: rule never runs, even if the engine considers it healthy. + * Use case: emergency kill-switch for a rule producing bad + * suggestions in production. + * + * File schema (JSON): + * { + * "version": 1, + * "force_enable": { "": { "ts": "", "reason": "" } }, + * "force_disable": { "": { "ts": "", "reason": "" } } + * } + * + * Reads are tolerant: a missing file → empty overrides. A malformed file is + * logged (via the caller's log hook) and treated as empty, never thrown — + * the dashboard must stay reachable even if someone hand-edited the JSON. + * Writes are atomic (temp + rename) so a crash mid-save can't leave the file + * half-written. + */ + +import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; + +const FILE_VERSION = 1; +const FILE_NAME = 'rule-overrides.json'; + +function overridesPath(projectDir) { + return join(projectDir, '.pos-supervisor', FILE_NAME); +} + +function emptyState() { + return { version: FILE_VERSION, force_enable: {}, force_disable: {} }; +} + +/** + * Load overrides from disk. Never throws — on any error returns empty state + * and calls `log` if provided. Intentional: a corrupt overrides file must + * not prevent the server from starting. + */ +export function loadOverrides(projectDir, { log } = {}) { + const path = overridesPath(projectDir); + if (!existsSync(path)) return emptyState(); + try { + const raw = readFileSync(path, 'utf-8'); + const parsed = JSON.parse(raw); + const fe = parsed?.force_enable ?? {}; + const fd = parsed?.force_disable ?? {}; + if (typeof fe !== 'object' || typeof fd !== 'object') { + throw new Error('force_enable / force_disable must be objects'); + } + return { version: FILE_VERSION, force_enable: { ...fe }, force_disable: { ...fd } }; + } catch (e) { + log?.(`rule-overrides: failed to parse ${path} (${e.message}); treating as empty`); + return emptyState(); + } +} + +/** + * Atomic write: stage to a sibling temp file, then rename. fs rename within + * the same dir is atomic on POSIX. A reader during the write sees either the + * old file or the new — never a torn read. + */ +export function saveOverrides(projectDir, state, { log } = {}) { + const path = overridesPath(projectDir); + mkdirSync(dirname(path), { recursive: true }); + const payload = JSON.stringify({ + version: FILE_VERSION, + force_enable: state.force_enable ?? {}, + force_disable: state.force_disable ?? {}, + }, null, 2); + const tmp = `${path}.tmp.${process.pid}.${Math.random().toString(36).slice(2)}`; + try { + writeFileSync(tmp, payload); + renameSync(tmp, path); + } catch (e) { + log?.(`rule-overrides: save failed (${e.message})`); + throw e; + } +} + +/** + * Register a force-enable for `ruleId`. Removes any force-disable for the + * same rule (the two are mutually exclusive — setting one clears the other). + * Persists immediately. + */ +export function addForceEnable(projectDir, ruleId, reason = '', { log } = {}) { + if (!ruleId) throw new Error('addForceEnable: ruleId required'); + const state = loadOverrides(projectDir, { log }); + state.force_enable[ruleId] = { ts: new Date().toISOString(), reason }; + delete state.force_disable[ruleId]; + saveOverrides(projectDir, state, { log }); + return state; +} + +export function addForceDisable(projectDir, ruleId, reason = '', { log } = {}) { + if (!ruleId) throw new Error('addForceDisable: ruleId required'); + const state = loadOverrides(projectDir, { log }); + state.force_disable[ruleId] = { ts: new Date().toISOString(), reason }; + delete state.force_enable[ruleId]; + saveOverrides(projectDir, state, { log }); + return state; +} + +export function removeOverride(projectDir, ruleId, { log } = {}) { + if (!ruleId) throw new Error('removeOverride: ruleId required'); + const state = loadOverrides(projectDir, { log }); + delete state.force_enable[ruleId]; + delete state.force_disable[ruleId]; + saveOverrides(projectDir, state, { log }); + return state; +} + +/** Convenience for callers that just want two string[] of rule_ids. */ +export function overrideSets(state) { + return { + force_enable: new Set(Object.keys(state.force_enable ?? {})), + force_disable: new Set(Object.keys(state.force_disable ?? {})), + }; +} diff --git a/src/core/rules/ConvertIncludeToRender.js b/src/core/rules/ConvertIncludeToRender.js new file mode 100644 index 0000000..0bd2e32 --- /dev/null +++ b/src/core/rules/ConvertIncludeToRender.js @@ -0,0 +1,30 @@ +/** + * ConvertIncludeToRender rule — migrate deprecated `{% include %}` to + * `{% render %}`. The LSP reports the check on every non-module include; this + * rule attaches the recommendation and stable attribution. A `text_edit` + * replacement (`include` → `render`) is produced by the heuristic + * fix-generator in full mode, with a guarded fallback for module-helper + * includes where the rename is NOT safe. + * + * The module-helper guard lives in fix-generator.js (pattern: `include + * 'modules/...'`). Keeping the skip logic there rather than duplicating it + * here means one source of truth — this rule just makes sure every include + * call gets a useful hint and a non-`unmatched` rule_id. + * + * Plan reference: Tier 1 trivial wins. + */ + +export const rules = [ + { + id: 'ConvertIncludeToRender.default', + check: 'ConvertIncludeToRender', + priority: 100, + when: () => true, + apply: () => ({ + rule_id: 'ConvertIncludeToRender.default', + hint_md: 'Replace `{% include "partial" %}` with `{% render "partial" %}`. `render` has isolated scope — pass every variable the partial needs explicitly: `{% render "partial", var: value %}`. Exception: `include` for **module helpers** (authorization, redirects) is correct because those partials intentionally need shared scope; the heuristic fix-generator detects this and proposes guidance instead of a rename.', + fixes: [], + confidence: 0.9, + }), + }, +]; diff --git a/src/core/rules/DeprecatedTag.js b/src/core/rules/DeprecatedTag.js new file mode 100644 index 0000000..ef01d27 --- /dev/null +++ b/src/core/rules/DeprecatedTag.js @@ -0,0 +1,139 @@ +/** + * DeprecatedTag rules — both the upstream LSP `DeprecatedTag` check (emitted + * by pos-cli for `{% include %}`, `{% parse_json %}`, etc.) AND the + * pos-supervisor structural variant `pos-supervisor:DeprecatedTag` (raised + * by `structural-warnings.detectDeprecatedTags` when the upstream check is + * silent for a tag we still want flagged). + * + * Rule_id pinning: every emit of either check now lands as + * `DeprecatedTag.` (or `.default`) in analytics. Before this module both + * checks were `*.unmatched`, so adoption + regression rates were collapsed + * across very different tags (`include` is ~100 % auto-fixable, `hash_assign` + * needs careful per-line edits, `parse_json` needs filter-syntax migration). + * + * Fix policy: + * - `include` → `text_edit` is owned by `ConvertIncludeToRender` (different + * check name, sibling rule). The deprecated-tag rule + * emits `guidance` only — duplicating the rename here + * would compete with that module. + * - `hash_assign` → fix-generator's `fixDeprecatedTag` produces a + * `hash_assign` → `assign` literal text_edit. We emit + * `guidance` so it isn't a duplicate; the heuristic + * edit complements the explanation. + * - `parse_json` → no live text_edit yet (the `| parse_json` filter form + * is structural, not a single-token swap). `guidance` + * plus an explicit migration recipe is the best signal + * short of an AST rewrite. + * - default → fallback for any other deprecated tag the upstream + * adds in future without a dedicated subrule. + */ + +const RULES_FOR_CHECK = (checkName) => [ + { + id: `${ruleIdPrefix(checkName)}.include`, + check: checkName, + priority: 10, + when: (diag) => /\binclude\b/.test(diag.params?.tag ?? '') || /\binclude\b/.test(diag.message ?? ''), + apply: () => ({ + rule_id: `${ruleIdPrefix(checkName)}.include`, + hint_md: + '`{% include %}` is deprecated. Replace with `{% render %}` everywhere in this file. ' + + '`{% render %}` has **isolated scope** — variables from the parent are NOT inherited; ' + + 'pass each one explicitly: `{% render \'partial\', name: name, items: items %}`. ' + + 'Exception: includes that pull in a module helper meant to share scope (auth, redirects) — ' + + 'leave those alone; the heuristic fix-generator detects this pattern and proposes guidance ' + + 'instead of a rename.', + fixes: [{ + type: 'guidance', + description: + 'Rename every `{% include \'X\' %}` in this file to `{% render \'X\', %}`. ' + + 'List the partial\'s declared `@param` names and pass each explicitly — ' + + 'isolated scope means undeclared vars come through as `nil`.', + }], + confidence: 0.95, + }), + }, + + { + id: `${ruleIdPrefix(checkName)}.hash_assign`, + check: checkName, + priority: 10, + when: (diag) => /hash_assign/.test(diag.params?.tag ?? '') || /\bhash_assign\b/.test(diag.message ?? ''), + apply: () => ({ + rule_id: `${ruleIdPrefix(checkName)}.hash_assign`, + hint_md: + '`{% hash_assign x, key: value %}` is deprecated. Use the bracket-assign form ' + + '`{% assign x["key"] = value %}` (or dot form `{% assign x.key = value %}` for ' + + 'identifier keys). Both produce the same hash — the new form is plain `assign`.', + fixes: [{ + type: 'guidance', + description: + 'Replace `{% hash_assign x, key: value %}` with `{% assign x["key"] = value %}`. ' + + 'For dotted identifiers: `{% assign x.key = value %}`. ' + + 'The heuristic fix-generator emits the literal `hash_assign` → `assign` text_edit; ' + + 'apply it then update the argument shape from `, key: value` to `["key"] = value`.', + }], + confidence: 0.85, + }), + }, + + { + id: `${ruleIdPrefix(checkName)}.parse_json`, + check: checkName, + priority: 10, + when: (diag) => /parse_json/.test(diag.params?.tag ?? '') || /\bparse_json\b/.test(diag.message ?? ''), + apply: () => ({ + rule_id: `${ruleIdPrefix(checkName)}.parse_json`, + hint_md: + '`{% parse_json x %}…{% endparse_json %}` is deprecated. Use the filter form ' + + '`{% assign x = \'\' | parse_json %}` (single-line) or build the literal with ' + + '`{% capture json %}…{% endcapture %}{% assign x = json | parse_json %}` for ' + + 'multi-line payloads. The filter is exact-equivalent semantically.', + fixes: [{ + type: 'guidance', + description: + 'Migrate `{% parse_json x %}{ … }{% endparse_json %}` to ' + + '`{% assign x = \'{ … }\' | parse_json %}`. For multi-line payloads use ' + + '`{% capture body %}{ … }{% endcapture %}{% assign x = body | parse_json %}` so the ' + + 'JSON is left literal — `parse_json` accepts strings, not block contents.', + }], + confidence: 0.85, + }), + }, + + { + id: `${ruleIdPrefix(checkName)}.default`, + check: checkName, + priority: 100, + when: () => true, + apply: (diag) => { + const tag = diag.params?.tag ?? null; + const replacement = diag.params?.replacement ?? null; + const tagSpan = tag ? `\`{% ${tag} %}\`` : 'this tag'; + const replSpan = replacement ? `Use \`{% ${replacement} %}\` instead.` : ''; + return { + rule_id: `${ruleIdPrefix(checkName)}.default`, + hint_md: + `${tagSpan} is deprecated in platformOS. ${replSpan} ` + + `Read the upstream message — it usually names the replacement and behavioral changes ` + + `(e.g. isolated scope for \`render\`). Fix every occurrence in this file in one pass; ` + + `leaving a single instance re-fires the check on the next write.`, + fixes: [], + confidence: 0.7, + }; + }, + }, +]; + +function ruleIdPrefix(checkName) { + // `pos-supervisor:DeprecatedTag` → rule_id starts with the bare check name + // so analytics aggregation across the upstream + structural variants stays + // readable. Storing rule_id as `pos-supervisor:DeprecatedTag.include` would + // break a couple of dashboard regexes that strip the colon segment. + return checkName.replace(/^pos-supervisor:/, ''); +} + +export const rules = [ + ...RULES_FOR_CHECK('DeprecatedTag'), + ...RULES_FOR_CHECK('pos-supervisor:DeprecatedTag'), +]; diff --git a/src/core/rules/DuplicateFunctionArguments.js b/src/core/rules/DuplicateFunctionArguments.js new file mode 100644 index 0000000..6a85c2f --- /dev/null +++ b/src/core/rules/DuplicateFunctionArguments.js @@ -0,0 +1,32 @@ +/** + * DuplicateFunctionArguments rule — pos-cli 6.0.7 duplicate-arg detection. + * + * Upstream fires for both `{% render %}` and `{% function %}` when the same + * argument name appears twice in a single call. Liquid's last-key-wins + * semantics silently drops the first value, so the bug is usually a typo + * (two args meant to be different but typed the same). The rule attaches + * a rule_id and an action-oriented hint; upstream supplies the autofix. + */ + +export const rules = [ + { + id: 'DuplicateFunctionArguments.default', + check: 'DuplicateFunctionArguments', + priority: 100, + when: () => true, + apply: (diag) => { + const arg = diag.params?.argument ?? 'the duplicate argument'; + const tag = diag.params?.tag_kind ?? 'render'; + const partial = diag.params?.partial ?? '(unknown partial)'; + return { + rule_id: 'DuplicateFunctionArguments.default', + hint_md: `\`${arg}\` is passed twice to \`{% ${tag} '${partial}' %}\`. Liquid keeps the LAST occurrence and silently drops earlier ones — usually this is a typo (you meant two different keys). Decide: same value? delete one. Different values intended? rename one to its real key.`, + fixes: [{ + type: 'guidance', + description: `Open the \`{% ${tag} '${partial}' %}\` call. Remove the second \`${arg}: …\` occurrence, or rename it to whatever distinct key was meant.`, + }], + confidence: 0.9, + }; + }, + }, +]; diff --git a/src/core/rules/GraphQLCheck.js b/src/core/rules/GraphQLCheck.js new file mode 100644 index 0000000..b6df7f2 --- /dev/null +++ b/src/core/rules/GraphQLCheck.js @@ -0,0 +1,137 @@ +/** + * GraphQLCheck rules — GraphQL validation errors. + * + * Priority order: + * 10 — unknown_field: field doesn't exist on type → schema-aware suggestions + * 20 — unused_variable: variable declared but never used in query + * 30 — type_mismatch: variable type doesn't match expected type + * 100 — generic: fallback hint + */ +import { nearestByLevenshtein } from './queries.js'; + +export const rules = [ + { + id: 'GraphQLCheck.unknown_field', + check: 'GraphQLCheck', + priority: 10, + when: (diag) => { + const cat = diag.params?.category; + return cat === 'unknown_field_record' || cat === 'unknown_field_other'; + }, + apply: (diag, facts) => { + const field = diag.params.field; + const typeName = diag.params.type; + const isRecord = diag.params.category === 'unknown_field_record'; + + if (isRecord && facts.graph) { + const schemaNodes = facts.graph.nodesByType('schema'); + const allProps = []; + for (const schema of schemaNodes) { + if (schema.properties) { + for (const p of schema.properties) { + if (p.name) allProps.push(p.name); + } + } + } + + const nearest = nearestByLevenshtein(field, allProps, 3); + const suggestion = nearest.length > 0 + ? `Did you mean \`${nearest[0].name}\`?` + : null; + const schemaList = schemaNodes.slice(0, 5).map(s => `\`${s.key}\``).join(', '); + + return { + rule_id: 'GraphQLCheck.unknown_field', + hint_md: `Cannot query field \`${field}\` on type \`${typeName}\`. In platformOS, Record fields come from schema definitions (tables).${suggestion ? ` ${suggestion}` : ''}\n\nAvailable tables: ${schemaList}${schemaNodes.length > 5 ? ` (+${schemaNodes.length - 5} more)` : ''}. Use \`properties\` to access custom fields, not top-level field names.`, + fixes: [], + confidence: 0.85, + see_also: { + tool: 'domain_guide', + args: { domain: 'graphql', section: 'gotchas' }, + reason: 'Unknown field on Record type. domain_guide(graphql, gotchas) explains how to query schema properties via the properties hash.', + }, + }; + } + + return { + rule_id: 'GraphQLCheck.unknown_field', + hint_md: `Cannot query field \`${field}\` on type \`${typeName}\`. Check the GraphQL schema for valid fields on this type.`, + fixes: [], + confidence: 0.6, + }; + }, + }, + + { + id: 'GraphQLCheck.unused_variable', + check: 'GraphQLCheck', + priority: 20, + when: (diag) => diag.params?.category === 'unused_variable', + apply: (diag) => { + const varName = diag.params.variable; + return { + rule_id: 'GraphQLCheck.unused_variable', + hint_md: `Variable \`$${varName}\` is declared but never used in the query. Either remove the variable declaration or use it in the query body.\n\nCommon causes: leftover from refactoring, copy-paste from another query, variable intended for a filter that was removed.`, + fixes: [], + confidence: 0.9, + }; + }, + }, + + { + id: 'GraphQLCheck.type_mismatch', + check: 'GraphQLCheck', + priority: 30, + when: (diag) => { + const cat = diag.params?.category; + return cat === 'type_mismatch_filter' || cat === 'type_mismatch_other'; + }, + apply: (diag) => { + const variable = diag.params.variable; + const actualType = diag.params.actual_type; + const expectedType = diag.params.expected_type; + const isFilter = diag.params.category === 'type_mismatch_filter'; + + if (isFilter) { + return { + rule_id: 'GraphQLCheck.type_mismatch', + hint_md: `Type mismatch: expected \`${expectedType}\` but got \`${actualType}\`.${variable ? ` Variable: \`$${variable}\`.` : ''}\n\nplatformOS uses filter types like \`StringFilter\`, \`IntFilter\`, etc. Instead of passing a plain value, wrap it: \`{ value: "your_value" }\` or \`{ value_in: ["a", "b"] }\`.`, + fixes: [], + confidence: 0.85, + see_also: { + tool: 'domain_guide', + args: { domain: 'graphql', section: 'gotchas' }, + reason: 'Filter type mismatch. domain_guide(graphql, gotchas) explains platformOS filter input types and how to construct them.', + }, + }; + } + + return { + rule_id: 'GraphQLCheck.type_mismatch', + hint_md: `Type mismatch: expected \`${expectedType}\` but got \`${actualType}\`.${variable ? ` Variable: \`$${variable}\`.` : ''} Check that the variable type in the query header matches the schema definition.`, + fixes: [], + confidence: 0.7, + }; + }, + }, + + { + id: 'GraphQLCheck.generic', + check: 'GraphQLCheck', + priority: 100, + when: () => true, + apply: (diag) => { + return { + rule_id: 'GraphQLCheck.generic', + hint_md: 'GraphQL validation error. Check the query for syntax errors, undefined variables, or field name typos. Use `domain_guide(graphql)` for platformOS-specific GraphQL conventions.', + fixes: [], + confidence: 0.4, + see_also: { + tool: 'domain_guide', + args: { domain: 'graphql' }, + reason: 'GraphQL error. domain_guide(graphql) covers platformOS-specific GraphQL patterns, filter types, and common mistakes.', + }, + }; + }, + }, +]; diff --git a/src/core/rules/GraphQLVariablesCheck.js b/src/core/rules/GraphQLVariablesCheck.js new file mode 100644 index 0000000..07a7104 --- /dev/null +++ b/src/core/rules/GraphQLVariablesCheck.js @@ -0,0 +1,239 @@ +/** + * GraphQLVariablesCheck rules — `{% graphql result = 'op_name', $X: Y %}` + * passed (or omitted) a variable that doesn't match the .graphql operation's + * declared signature. + * + * Pre-rule the check landed as `.unmatched` (3 emits in DEMO, 100 % + * resolution but 0 % adoption — the LSP message named the variable but + * carried no actionable fix). The rule extracts variable + direction + * (required vs unknown) from the message and, when the file has graphql + * calls indexed by the project graph, surfaces the operation's full + * variable signature so the agent can pick the right value type. + * + * Subrules: + * • GraphQLVariablesCheck.parser_blind_spot — call lives inside a + * `{% liquid %}` block with multi-line `,` continuation. Both + * liquid-html-parser and pos-cli's LSP truncate the call at the first + * newline-comma; LSP fires `.required` for every arg past it. The + * agent sees the args in source, our default `.required` hint says + * "add the arg" — agent enters a regression spiral. This sub-rule + * fires first when the project graph reports the file's graphql call + * with `source_kind === 'liquid_multiline_truncated'` and steers the + * agent at the syntactic root cause instead. (Reproduced in DEMO + * 2026-04-27, 4 emits / 100 % regression.) + * • GraphQLVariablesCheck.required — agent forgot a `$var` argument. + * • GraphQLVariablesCheck.unknown — agent passed an undeclared `$var`. + * • GraphQLVariablesCheck.default — extractor failed; bare hint. + * + * Fix policy: guidance-only; the deterministic edit needs the call's + * argument list which the rule layer doesn't have. + */ + +export const rules = [ + { + id: 'GraphQLVariablesCheck.parser_blind_spot', + check: 'GraphQLVariablesCheck', + priority: 3, + when: (diag, facts) => isParserBlindSpot(diag, facts), + apply: (diag, facts) => buildParserBlindSpotHint(diag, facts), + }, + { + id: 'GraphQLVariablesCheck.required', + check: 'GraphQLVariablesCheck', + priority: 5, + when: (diag) => diag.params?.direction === 'required', + apply: (diag, facts) => buildRequiredHint(diag, facts), + }, + { + id: 'GraphQLVariablesCheck.unknown', + check: 'GraphQLVariablesCheck', + priority: 6, + when: (diag) => diag.params?.direction === 'unknown', + apply: (diag, facts) => buildUnknownHint(diag, facts), + }, + { + id: 'GraphQLVariablesCheck.default', + check: 'GraphQLVariablesCheck', + priority: 100, + when: () => true, + apply: (diag) => ({ + rule_id: 'GraphQLVariablesCheck.default', + hint_md: + `\`{% graphql %}\` variable mismatch. Open the called .graphql operation under \`app/graphql/\` ` + + `and read the operation header — variables declared as \`$name: Type\` (no leading \`$\` is wrong). ` + + `Required → add the argument to the tag (\`{% graphql r = 'op', name: value %}\`); Unknown → drop it.`, + fixes: [{ + type: 'guidance', + description: + `Open the .graphql file's operation header to see the full \`$variable: Type\` signature, then ` + + `pass each required variable as a named argument on the \`{% graphql %}\` tag.`, + }], + confidence: 0.5, + }), + }, +]; + +/** + * True when the diagnostic looks like the multi-line truncation false-flag: + * • LSP fired `direction: required` (it sees no args at all). + * • Project graph has a graphql call from this file whose extracted + * `source_kind === 'liquid_multiline_truncated'`. + * + * `source_kind` is populated by `liquid-parser.classifyGraphqlSourceKind` + * during scan — see liquid-parser.js for the detection criterion. Falsy + * graphs / unindexed files / unrelated source kinds all fall through to the + * downstream `.required` rule, so this predicate is purely additive. + */ +function isParserBlindSpot(diag, facts) { + if (diag?.params?.direction !== 'required') return false; + const node = facts?.graph?.nodeByPath?.(diag?.file); + const calls = node?.graphql_calls ?? []; + return calls.some(c => c?.source_kind === 'liquid_multiline_truncated'); +} + +function buildParserBlindSpotHint(diag, facts) { + const param = diag?.params?.param_name ?? ''; + const sigBlock = signatureBlock(diag, facts); + return { + rule_id: 'GraphQLVariablesCheck.parser_blind_spot', + hint_md: + `\`{% graphql %}\` call appears to pass \`${param}\`, but the parser cannot see it. ` + + `The call lives inside a \`{% liquid %}\` block written with multi-line \`,\` ` + + `continuation — both pos-cli's check and the AST parser stop at the first newline-comma, ` + + `so every named argument past it is silently dropped.\n\n` + + `Do NOT keep adding the argument — it is already there in source. **Fix the syntax**:\n\n` + + '```liquid\n' + + `{% graphql result = '', ${param}: ${param}, ... %} # tag form, args on one line\n` + + '```\n' + + `or, if you must keep it inside \`{% liquid %}\`, put every named arg on the SAME line as ` + + `\`graphql\`:\n\n` + + '```liquid\n' + + `{% liquid\n` + + ` graphql result = '', ${param}: ${param}, email: email, ...\n` + + `%}\n` + + '```' + + sigBlock, + fixes: [{ + type: 'guidance', + description: + `Convert the multi-line \`graphql\` call to a single-line form. Either move it out of ` + + `\`{% liquid %}\` into \`{% graphql ... %}\` tag delimiters, or keep it inside the block ` + + `but place every \`name: value\` argument on the same line as \`graphql\`. The arguments ` + + `you wrote are correct — only the line breaks are dropping them.${diagFiles(diag, facts)}`, + }], + confidence: 0.95, + see_also: { + tool: 'domain_guide', + args: { domain: 'graphql' }, + reason: 'Multi-line `{% graphql %}` continuation inside `{% liquid %}` is silently truncated. domain_guide(graphql) shows the canonical tag form.', + }, + }; +} + +function buildRequiredHint(diag, facts) { + const param = diag.params?.param_name ?? ''; + const sigBlock = signatureBlock(diag, facts); + return { + rule_id: 'GraphQLVariablesCheck.required', + hint_md: + `\`{% graphql %}\` call is missing required variable \`${param}\`. The operation declares ` + + `\`$${param}: \` in its header — every non-optional variable (no \`= default\`) MUST be passed ` + + `at the call site.\n\n` + + `Add to the tag:\n` + + '```liquid\n' + + `{% graphql result = '', ${param}: ${param} %} # forward caller scope\n` + + `{% graphql result = '', ${param}: \"value\" %} # literal\n` + + `{% graphql result = '', ${param}: context.params.${param} %} # request param\n` + + '```' + + sigBlock, + fixes: [{ + type: 'guidance', + description: + `Add \`${param}: \` to the \`{% graphql %}\` tag. The value must match the declared ` + + `GraphQL type — pass a string for \`String!\`, an integer for \`Int!\`, an object literal for ` + + `input types, etc.${diagFiles(diag, facts)}`, + }], + confidence: 0.75, + see_also: { + tool: 'domain_guide', + args: { domain: 'graphql' }, + reason: 'GraphQL call variable mismatch. domain_guide(graphql) covers $variable signatures and value forwarding.', + }, + }; +} + +function buildUnknownHint(diag, facts) { + const param = diag.params?.param_name ?? ''; + const sigBlock = signatureBlock(diag, facts); + return { + rule_id: 'GraphQLVariablesCheck.unknown', + hint_md: + `\`{% graphql %}\` call passes \`${param}\` but the operation does NOT declare \`$${param}\`. ` + + `Undeclared variables are silently dropped at call time — this is dead data that may mask a typo.\n\n` + + `Pick one fix:\n` + + ` A) **Drop** \`${param}: ...\` from the \`{% graphql %}\` tag in this file.\n` + + ` B) **Declare** \`$${param}: \` in the .graphql operation's variable list (and use it in ` + + `the body — orphan declarations themselves trigger \`GraphQLCheck\`).\n` + + ` C) **Rename** \`${param}\` to match an existing operation variable — common cause is a typo.` + + sigBlock, + fixes: [{ + type: 'guidance', + description: + `Pick: (A) drop \`${param}: \` from the call, (B) add \`$${param}: \` to the .graphql ` + + `operation header, or (C) rename \`${param}\` to a declared variable.${diagFiles(diag, facts)}`, + }], + confidence: 0.75, + see_also: { + tool: 'domain_guide', + args: { domain: 'graphql' }, + reason: 'GraphQL call passes an undeclared variable. domain_guide(graphql) covers $variable signatures.', + }, + }; +} + +/** + * Build a markdown block listing the declared variables of every graphql + * operation called from `diag.file`. Empty string when the file is not + * indexed or has no graphql_calls. + * + * Uses the graph's `graphql_calls` (which carries `{ variable, queryName }` + * per call) and the per-operation node's `args` list (parsed from the + * `.graphql` file's `query Foo($x: String!) { ... }` header). + */ +function signatureBlock(diag, facts) { + const sigs = collectSignatures(diag, facts); + if (sigs.length === 0) return ''; + const list = sigs.map(s => { + const args = s.args.length === 0 + ? '(no variables)' + : s.args.map(a => `\`$${a.name}: ${a.type}\``).join(', '); + return ` • \`${s.queryName}\` — ${args}`; + }).join('\n'); + return `\n\nGraphQL operation(s) called from this file:\n${list}`; +} + +function diagFiles(diag, facts) { + const sigs = collectSignatures(diag, facts); + if (sigs.length !== 1) return ''; + return ` Reference: \`app/graphql/${sigs[0].queryName}.graphql\`.`; +} + +function collectSignatures(diag, facts) { + const graph = facts?.graph; + const filePath = diag?.file; + if (!graph || !filePath) return []; + const node = graph.nodeByPath(filePath); + if (!node) return []; + const calls = node.graphql_calls ?? []; + const out = []; + const seen = new Set(); + for (const call of calls) { + const queryName = typeof call === 'string' ? call : call?.queryName; + if (!queryName || seen.has(queryName)) continue; + seen.add(queryName); + const opNode = graph.nodeByKey('graphql', queryName); + if (!opNode) continue; + out.push({ queryName, args: opNode.args ?? [] }); + } + return out; +} diff --git a/src/core/rules/ImgLazyLoading.js b/src/core/rules/ImgLazyLoading.js new file mode 100644 index 0000000..1fc8b65 --- /dev/null +++ b/src/core/rules/ImgLazyLoading.js @@ -0,0 +1,24 @@ +/** + * ImgLazyLoading rule — performance hint. The LSP flags an `` without + * `loading="lazy"`; this rule attaches the recommendation text and stable + * attribution. The `text_edit` insert itself is produced by the heuristic + * fix-generator in full mode — rules this trivial don't need to reimplement + * position calculation. + * + * Plan reference: Tier 1 trivial wins. + */ + +export const rules = [ + { + id: 'ImgLazyLoading.recommended', + check: 'ImgLazyLoading', + priority: 100, + when: () => true, + apply: () => ({ + rule_id: 'ImgLazyLoading.recommended', + hint_md: 'Add `loading="lazy"` to this `` tag so the browser defers off-screen image loads. Improves Core Web Vitals (LCP) and reduces initial bytes transferred on long pages.', + fixes: [], + confidence: 0.9, + }), + }, +]; diff --git a/src/core/rules/ImgWidthAndHeight.js b/src/core/rules/ImgWidthAndHeight.js new file mode 100644 index 0000000..3a5f631 --- /dev/null +++ b/src/core/rules/ImgWidthAndHeight.js @@ -0,0 +1,24 @@ +/** + * ImgWidthAndHeight rule — layout-shift hint. The LSP flags an `` without + * explicit `width` and/or `height`; this rule attaches guidance and stable + * attribution. The `text_edit` insert is produced by the heuristic + * fix-generator (full mode); the rule keeps quick-mode diagnostics usefully + * labelled. + * + * Plan reference: Tier 1 trivial wins. + */ + +export const rules = [ + { + id: 'ImgWidthAndHeight.recommended', + check: 'ImgWidthAndHeight', + priority: 100, + when: () => true, + apply: () => ({ + rule_id: 'ImgWidthAndHeight.recommended', + hint_md: 'Add explicit `width` and `height` attributes to this `` tag. The browser uses them to reserve space before the image loads, eliminating cumulative layout shift (CLS). For responsive images, set the attributes to the intrinsic dimensions and override with CSS (`style="width:100%;height:auto"`).', + fixes: [], + confidence: 0.9, + }), + }, +]; diff --git a/src/core/rules/InvalidLayout.js b/src/core/rules/InvalidLayout.js new file mode 100644 index 0000000..d811866 --- /dev/null +++ b/src/core/rules/InvalidLayout.js @@ -0,0 +1,74 @@ +/** + * pos-supervisor:InvalidLayout rule — page front-matter references a layout + * that doesn't exist on disk. + * + * Pre-rule the check landed as `.unmatched` even though fix-generator's + * `fixStructuralCheck` already produced a `create_file` proposal. Pre-task-4 + * the proposal hardcoded the `.html.liquid` extension which DOES NOT match + * many projects (DEMO included) — agents accepted the fix, the file landed + * at the wrong path, and the original error never resolved. + * + * Task 4 fixed the path in the structural emitter (it now embeds the right + * extension via `detectLayoutExtension`) and made `extractLayoutPath` + * lift the path verbatim from the message. This rule attaches stable + * attribution + a guidance fix that explains the two valid resolutions + * (rename the layout reference vs create the missing layout file). + * + * Pairs with `ValidFrontmatter.layout_missing` (upstream LSP). The dedup + * pass `suppressUpstreamFrontmatterDup` drops the upstream copy so only + * this rule fires per offending page. + */ + +const MSG_RE = /^Layout `([^`]+)` not found\. Expected file: `([^`]+)`\./; + +export const rules = [ + { + id: 'InvalidLayout.default', + check: 'pos-supervisor:InvalidLayout', + priority: 100, + when: () => true, + apply: (diag) => { + const m = (diag.message ?? '').match(MSG_RE); + const layoutName = m?.[1] ?? null; + const expectedPath = m?.[2] ?? null; + + const layoutSpan = layoutName ? `\`${layoutName}\`` : 'the referenced layout'; + const pathSpan = expectedPath ? `\`${expectedPath}\`` : 'the expected layout file'; + + const hint = + `${layoutSpan} is not on disk. The structural emitter resolved the canonical path to ` + + `${pathSpan} (extension picked to match the project's existing layouts).\n\n` + + `Two resolutions:\n` + + ` • **Layout reference is wrong** — fix \`layout: ${layoutName ?? ''}\` in this page's ` + + `front matter. Run \`project_map\` to see which layouts exist.\n` + + ` • **Layout file is missing** — create ${pathSpan}. Every layout MUST contain ` + + `\`{{ content_for_layout }}\` exactly once (and may add named \`{% yield 'name' %}\` slots).`; + + return { + rule_id: 'InvalidLayout.default', + hint_md: hint, + fixes: expectedPath + ? [{ + type: 'create_file', + path: expectedPath, + description: + `Create ${pathSpan} with at least \`{{ content_for_layout }}\` so pages using ` + + `\`layout: ${layoutName ?? ''}\` render. Verify the layout name is intentional first — ` + + `a typo in the page front matter is a more common cause than a genuinely missing layout.`, + }] + : [{ + type: 'guidance', + description: + `Either fix the layout name in the page's front matter, or create the layout file. ` + + `Run \`project_map\` to see which layouts exist.`, + }], + confidence: 0.85, + see_also: { + tool: 'domain_guide', + args: { domain: 'layouts' }, + reason: 'Layout file conventions — required `{{ content_for_layout }}`, named yield slots, locations.', + }, + }; + }, + }, +]; diff --git a/src/core/rules/JsonLiteralQuoteStyle.js b/src/core/rules/JsonLiteralQuoteStyle.js new file mode 100644 index 0000000..3f06699 --- /dev/null +++ b/src/core/rules/JsonLiteralQuoteStyle.js @@ -0,0 +1,28 @@ +/** + * JsonLiteralQuoteStyle rule — pos-cli 6.0.7 inline JSON-literal grammar check. + * + * Upstream emits a single constant message when a single-quoted string appears + * inside a `{ … }` or `[ … ]` literal in `{% assign %}` / `{% return %}` / + * `{% function %}` arguments. The fix is mechanical: change the quotes to + * double quotes. The rule attaches a stable rule_id + a quote-swap-flavored + * hint; the upstream check itself ships an autofix corrector, so we don't + * duplicate the text_edit here. + */ + +export const rules = [ + { + id: 'JsonLiteralQuoteStyle.default', + check: 'JsonLiteralQuoteStyle', + priority: 100, + when: () => true, + apply: () => ({ + rule_id: 'JsonLiteralQuoteStyle.default', + hint_md: 'String literals inside `{ … }` or `[ … ]` JSON literals must be double-quoted. Change the offending single quote to a double quote — the rest of the literal is fine. Liquid string assigns outside JSON literals (`{% assign x = \'hi\' %}`) are not affected.', + fixes: [{ + type: 'guidance', + description: "Replace the single-quoted string with a double-quoted equivalent. Example: `{ 'k': 'v' }` → `{ \"k\": \"v\" }`. The upstream check ships an autofix the agent can accept directly.", + }], + confidence: 0.95, + }), + }, +]; diff --git a/src/core/rules/LiquidHTMLSyntaxError.js b/src/core/rules/LiquidHTMLSyntaxError.js new file mode 100644 index 0000000..cbd340a --- /dev/null +++ b/src/core/rules/LiquidHTMLSyntaxError.js @@ -0,0 +1,166 @@ +/** + * LiquidHTMLSyntaxError rules — pos-cli surfaces a varied set of parser + * errors under one check name: + * - Unknown tag + * - For-loop argument shape mismatch + * - Missing `=` in graphql/function assigns (heuristic owns text_edit) + * - Inline array/hash literal in tag arguments + * - Unclosed Liquid block / mismatched quotes + * + * Pre-rule every emit landed as `.unmatched` even though the messages + * carry a clear discriminator. The rule routes by message shape and emits + * subrule-specific guidance so analytics distinguish the very different + * resolution rates (unknown_tag at ~100 % vs nested array literal where the + * agent often misreads the cause). + * + * Fix policy: + * - missing_assign — heuristic in fix-generator already produces a + * text_edit. Rule emits guidance; precedence drops the heuristic + * `guidance` if any (it isn't there in this branch). + * - All others — guidance only; no deterministic AST rewrite. + */ + +import { nearestByLevenshtein } from './queries.js'; + +export const rules = [ + { + id: 'LiquidHTMLSyntaxError.unknown_tag', + check: 'LiquidHTMLSyntaxError', + priority: 5, + when: (diag) => /Unknown tag/i.test(diag.message ?? ''), + apply: (diag, facts) => { + const m = diag.message.match(/Unknown tag\s+['"`]?(\w+)['"`]?/i); + const badTag = m?.[1] ?? null; + const tagsIndex = facts?.tagsIndex; + + let didYouMean = ''; + if (badTag && tagsIndex?.platformOSTags) { + const known = tagsIndex.platformOSTags().map(t => t.name); + const nearest = nearestByLevenshtein(badTag, known, 3); + if (nearest.length > 0) { + const list = nearest.map(n => `\`{% ${n.name} %}\``).join(', '); + didYouMean = ` Did you mean: ${list}?`; + } + } + + const tagSpan = badTag ? `\`{% ${badTag} %}\`` : 'this tag'; + return { + rule_id: 'LiquidHTMLSyntaxError.unknown_tag', + hint_md: + `${tagSpan} is not a recognised Liquid tag in platformOS.${didYouMean}\n\n` + + `Common causes: typo in the tag name, custom tag from another framework (Shopify Liquid extras ` + + `like \`{% layout %}\`, \`{% schema %}\` are NOT supported), or a stale rename. ` + + `If the tag is custom: platformOS does not support custom tags — restructure as a partial ` + + `(\`{% render %}\`) or filter.`, + fixes: [{ + type: 'guidance', + description: badTag + ? `Replace ${tagSpan} with the correct tag name (see suggestions in the hint), or ` + + `restructure if it was a Shopify-only tag.` + : `Read the upstream message — it names the unknown tag. Replace it with a valid platformOS ` + + `tag or restructure the logic.`, + }], + confidence: 0.85, + }; + }, + }, + + { + id: 'LiquidHTMLSyntaxError.for_loop_args', + check: 'LiquidHTMLSyntaxError', + priority: 10, + when: (diag) => /Arguments must be provided in the format `for in/i.test(diag.message ?? ''), + apply: (diag) => { + const m = diag.message.match(/Invalid\/Unknown arguments:\s*(.+)$/i); + const badArgs = m?.[1]?.trim() ?? null; + const argsSpan = badArgs ? `\`${badArgs}\`` : 'the offending argument(s)'; + return { + rule_id: 'LiquidHTMLSyntaxError.for_loop_args', + hint_md: + `\`{% for %}\` arguments must follow the form ` + + `\`for in [reversed] [limit:N] [offset:N]\`. ` + + `${argsSpan} ${badArgs ? 'are' : 'is'} not a recognised positional or named argument.\n\n` + + `Frequent root cause: a Liquid filter (\`| t\`, \`| split\`, etc.) appears INSIDE the ` + + `\`for ... in ...\` clause. The Liquid parser does not accept filter pipelines in the loop ` + + `header — assign the filtered value first, then iterate.\n\n` + + `Wrong: \`{% for item in 'k' | t %}\` Right: \`{% assign items = 'k' | t %}{% for item in items %}\`. ` + + `Wrong: \`{% for word in str | split: ',' %}\` Right: \`{% assign words = str | split: ',' %}{% for word in words %}\`.`, + fixes: [{ + type: 'guidance', + description: + `Move the filter pipeline out of the \`for in \` clause: ` + + `\`{% assign items = %}\` first, then \`{% for item in items %}\`. ` + + `For nested loops over translation arrays, see the TranslationKeyExists.array_index_misuse ` + + `pattern.`, + }], + confidence: 0.85, + }; + }, + }, + + { + id: 'LiquidHTMLSyntaxError.missing_assign', + check: 'LiquidHTMLSyntaxError', + priority: 15, + when: (diag) => /\{%\s*(?:graphql|function)/.test(diag.message ?? '') && /=/.test(diag.message ?? ''), + apply: () => ({ + rule_id: 'LiquidHTMLSyntaxError.missing_assign', + hint_md: + '`{% graphql %}` and `{% function %}` require an assignment target. The syntax is ' + + '`{% graphql result = \'query_name\' %}` and `{% function result = \'path/to/helper\', arg: val %}` — ' + + 'the `result =` part captures the call output and is not optional.', + fixes: [{ + type: 'guidance', + description: + 'Add ` =` between the tag name and the call path. ' + + '`{% graphql records = \'q\' %}` / `{% function record = \'helper\', x: 1 %}`. ' + + 'fix-generator emits the literal text_edit for the missing-`=` shape — accept it.', + }], + confidence: 0.9, + }), + }, + + { + id: 'LiquidHTMLSyntaxError.inline_literal', + check: 'LiquidHTMLSyntaxError', + priority: 20, + when: (diag) => /(?:array|hash|object|literal|inline)/i.test(diag.message ?? '') && + /\{%\s*(?:render|function|graphql)/.test(diag.message ?? ''), + apply: () => ({ + rule_id: 'LiquidHTMLSyntaxError.inline_literal', + hint_md: + 'Inline `[…]` array literals and `{ … }` hash literals are NOT accepted as tag arguments. ' + + 'Liquid\'s tag parser only takes named scalars and pre-assigned variables. Build the literal ' + + 'in a preceding `{% assign %}` then pass the variable.\n\n' + + 'Wrong: `{% render \'p\', items: [] %}` Right: `{% assign items = [] %}{% render \'p\', items: items %}`.', + fixes: [{ + type: 'guidance', + description: + 'Pre-assign the literal: `{% assign items = […] %}` (or `{% assign cfg = { … } %}` for hashes), ' + + 'then pass `items` (or `cfg`) by name in the render/function/graphql tag.', + }], + confidence: 0.85, + }), + }, + + { + id: 'LiquidHTMLSyntaxError.default', + check: 'LiquidHTMLSyntaxError', + priority: 100, + when: () => true, + apply: () => ({ + rule_id: 'LiquidHTMLSyntaxError.default', + hint_md: + 'Liquid parser error. Read the upstream message — it names the line and column. ' + + 'Common causes:\n' + + ' • Unclosed block (`{% if %}` without `{% endif %}`, `{% for %}` without `{% endfor %}`).\n' + + ' • Inside `{% liquid %}` blocks each statement is on its own line with NO delimiters.\n' + + ' • Mismatched quotes — every `\'` and `"` must be paired on the same logical token.\n' + + ' • HTML and Liquid syntax interleaved unsafely (e.g. `
` ' + + 'is fine; `
` is not).\n\n' + + 'Fix the FIRST reported error — later errors often cascade from it.', + fixes: [], + confidence: 0.5, + }), + }, +]; diff --git a/src/core/rules/MetadataParamsCheck.js b/src/core/rules/MetadataParamsCheck.js new file mode 100644 index 0000000..a668a47 --- /dev/null +++ b/src/core/rules/MetadataParamsCheck.js @@ -0,0 +1,99 @@ +/** + * MetadataParamsCheck rules — metadata/doc block parameter violations. + * + * Priority order: + * 10 — module_contract: partial belongs to a known module → show module API + * 20 — doc_block_params: cross-reference {% doc %} params with callers + * 100 — generic: fallback hint + */ + +export const rules = [ + { + id: 'MetadataParamsCheck.module_contract', + check: 'MetadataParamsCheck', + priority: 10, + when: (diag) => { + const msg = diag.message ?? ''; + const partialMatch = msg.match(/['"`]([^'"`]+)['"`]/); + const name = partialMatch?.[1]; + return !!name && name.startsWith('modules/'); + }, + apply: (diag) => { + const msg = diag.message ?? ''; + const partialMatch = msg.match(/['"`]([^'"`]+)['"`]/); + const name = partialMatch?.[1]; + const moduleName = name?.split('/')[1] ?? 'unknown'; + const isFunctionCall = diag.params?.is_function_call === 'true'; + const tag = isFunctionCall ? 'function' : 'render'; + + return { + rule_id: 'MetadataParamsCheck.module_contract', + hint_md: `Parameter mismatch on module ${tag} call \`${name}\`. Module partials define their contract via \`{% doc %}\` blocks — use \`module_info\` to see the expected signature.`, + fixes: [], + confidence: 0.85, + see_also: { + tool: 'module_info', + args: { name: moduleName, section: 'api' }, + reason: `Module '${moduleName}' param mismatch. module_info(${moduleName}, api) shows the full signature with required/optional params.`, + }, + }; + }, + }, + + { + id: 'MetadataParamsCheck.doc_block_params', + check: 'MetadataParamsCheck', + priority: 20, + when: (diag, facts) => { + const msg = diag.message ?? ''; + const partialMatch = msg.match(/['"`]([^'"`]+)['"`]/); + const name = partialMatch?.[1]; + if (!name || name.startsWith('modules/')) return false; + const sig = facts.graph.partialSignature(name); + return sig !== null && sig.length > 0; + }, + apply: (diag, facts) => { + const msg = diag.message ?? ''; + const partialMatch = msg.match(/['"`]([^'"`]+)['"`]/); + const name = partialMatch?.[1]; + const sig = facts.graph.partialSignature(name); + const isFunctionCall = diag.params?.is_function_call === 'true'; + const tag = isFunctionCall ? 'function' : 'render'; + + const paramList = sig.map(p => { + const req = p.required ? '(required)' : '(optional)'; + return `\`${p.name}\` ${req}`; + }).join(', '); + + return { + rule_id: 'MetadataParamsCheck.doc_block_params', + hint_md: `Parameter issue on \`{% ${tag} '${name}' %}\`. Declared params: ${paramList}.\n\nCheck the \`{% doc %}\` block in \`${name}\` for the full contract.`, + fixes: [], + confidence: 0.8, + see_also: { + tool: 'domain_guide', + args: { domain: 'partials', section: 'api' }, + reason: 'Render call param mismatch. domain_guide(partials, api) explains how {% doc %} @param declarations interact with render and function calls.', + }, + }; + }, + }, + + { + id: 'MetadataParamsCheck.generic', + check: 'MetadataParamsCheck', + priority: 100, + when: () => true, + apply: (diag) => { + const isFunctionCall = diag.params?.is_function_call === 'true'; + return { + rule_id: 'MetadataParamsCheck.generic', + hint_md: isFunctionCall + ? 'Function call parameter mismatch. Check the `{% doc %}` block in the target command/query for required `@param` declarations.' + : 'Render call parameter mismatch. Check the `{% doc %}` block in the target partial for required `@param` declarations.', + fixes: [], + confidence: 0.4, + }; + }, + }, +]; diff --git a/src/core/rules/MissingAsset.js b/src/core/rules/MissingAsset.js new file mode 100644 index 0000000..cf1f8d5 --- /dev/null +++ b/src/core/rules/MissingAsset.js @@ -0,0 +1,145 @@ +/** + * MissingAsset rules — `{{ 'foo.css' | asset_url }}` or + * `{% include_asset 'foo.js' %}` references a file that doesn't exist + * under `app/assets/`. + * + * Pre-rule the check landed as `.unmatched`; fix-generator's + * `fixMissingAsset` produced an unconditional `create_file` proposal + * (`app/assets/`). That's wrong roughly half the time — the more + * common case is a typo or a missing subdirectory prefix + * (`logo.png` vs `images/logo.png`). The DEMO data showed 0 % resolution, + * 33 % adoption-but-`partial` (agent ran the create_file then realized the + * intended file was elsewhere). + * + * Subrules: + * 5 — missing_subdir_prefix: bare filename like `logo.png` matches an + * existing asset under a known subdir (`images/logo.png`). + * Highest-confidence "this is a typo, fix the reference" signal. + * 10 — suggest_nearest: Levenshtein vs `assetNames(graph)`. Catches + * ordinary typos (`maain.css` → `main.css`). + * 100 — create_file: nothing close, propose creating it. Mirrors the + * existing heuristic so analytics gets stable rule_id even when no + * match exists. + */ + +import { assetNames, nearestByLevenshtein } from './queries.js'; + +const KNOWN_ASSET_SUBDIRS = ['images', 'styles', 'scripts', 'fonts', 'media']; + +export const rules = [ + { + id: 'MissingAsset.missing_subdir_prefix', + check: 'MissingAsset', + priority: 5, + when: (diag, facts) => { + const wanted = parseAssetPath(diag.message); + if (!wanted || wanted.includes('/')) return false; + const all = assetNames(facts?.graph); + return all.some(a => assetMatchesBareName(a, wanted)); + }, + apply: (diag, facts) => { + const wanted = parseAssetPath(diag.message); + const all = assetNames(facts.graph); + const matches = all.filter(a => assetMatchesBareName(a, wanted)); + const best = matches[0]; + const matchList = matches.slice(0, 5).map(m => `\`${m}\``).join(', '); + return { + rule_id: 'MissingAsset.missing_subdir_prefix', + hint_md: + `\`${wanted}\` is not at \`app/assets/${wanted}\` directly, but a file with this name lives under ` + + `a subdirectory: ${matchList}. \`asset_url\` paths are relative to \`app/assets/\` AND must include ` + + `the subdirectory (\`images/\`, \`styles/\`, \`scripts/\`, \`fonts/\`, \`media/\`). Fix the reference, ` + + `don't create a new file.`, + fixes: [{ + type: 'guidance', + description: + `Replace \`'${wanted}'\` with \`'${best}'\` in the \`asset_url\` filter (or \`include_asset\` tag). ` + + `Do NOT create a new \`app/assets/${wanted}\` — the file already exists at \`app/assets/${best}\`.`, + }], + confidence: 0.9, + }; + }, + }, + + { + id: 'MissingAsset.suggest_nearest', + check: 'MissingAsset', + priority: 10, + when: (diag, facts) => { + const wanted = parseAssetPath(diag.message); + if (!wanted) return false; + const all = assetNames(facts?.graph); + if (all.length === 0) return false; + return nearestByLevenshtein(wanted, all, 3).length > 0; + }, + apply: (diag, facts) => { + const wanted = parseAssetPath(diag.message); + const all = assetNames(facts.graph); + const nearest = nearestByLevenshtein(wanted, all, 3); + const best = nearest[0].name; + const list = nearest.map(n => `\`${n.name}\``).join(', '); + return { + rule_id: 'MissingAsset.suggest_nearest', + hint_md: + `\`${wanted}\` not found under \`app/assets/\`. Did you mean: ${list}? ` + + `If the reference is a typo, fix it. If you genuinely need a new asset, create the file ` + + `at \`app/assets/${wanted}\`.`, + fixes: [{ + type: 'guidance', + description: + `Replace \`'${wanted}'\` with \`'${best}'\` in the \`asset_url\` filter (or \`include_asset\` tag). ` + + `Distance ${nearest[0].distance} — verify the suggestion before applying.`, + }], + confidence: nearest[0].distance <= 2 ? 0.85 : 0.65, + }; + }, + }, + + { + id: 'MissingAsset.create_file', + check: 'MissingAsset', + priority: 100, + when: () => true, + apply: (diag) => { + const wanted = parseAssetPath(diag.message); + const targetPath = wanted ? `app/assets/${wanted}` : 'app/assets/'; + return { + rule_id: 'MissingAsset.create_file', + hint_md: + `\`${wanted ?? 'asset'}\` does not exist under \`app/assets/\`. ` + + `\`asset_url\` paths are relative to \`app/assets/\` AND must include the subdirectory ` + + `(\`images/\`, \`styles/\`, \`scripts/\`, \`fonts/\`, \`media/\`). ` + + `If this is a module-shipped asset the file may already exist inside the module's ` + + `\`public/assets/\` — module assets are referenced through the same \`asset_url\` filter ` + + `and should resolve automatically; if they don't, the module isn't installed.`, + fixes: [{ + type: 'guidance', + description: + `Create the asset at \`${targetPath}\`, OR (more likely) fix the reference — module assets ` + + `live under \`modules//public/assets/\` and resolve through the same \`asset_url\` filter ` + + `without any path prefix. Only create a new file when you control the asset and it is genuinely missing.`, + }], + confidence: 0.6, + }; + }, + }, +]; + +function parseAssetPath(message) { + if (!message) return null; + const m = message.match(/['"`]([^'"`]+)['"`]\s+does not exist/); + return m ? m[1] : null; +} + +function assetMatchesBareName(assetPath, bareName) { + // `assetPath` is relative to app/assets/, e.g. `images/logo.png`. We're + // matching when the LAST segment equals the bare name AND the leading + // segment is one of the conventional asset subdirs. This avoids + // false-positive matches against unrelated nested files + // (e.g. agent wrote `data.json` and we'd otherwise match `vendor/x/data.json`). + const slash = assetPath.indexOf('/'); + if (slash < 0) return false; + const subdir = assetPath.slice(0, slash); + const tail = assetPath.slice(slash + 1); + return tail === bareName && KNOWN_ASSET_SUBDIRS.includes(subdir); +} diff --git a/src/core/rules/MissingContentForLayout.js b/src/core/rules/MissingContentForLayout.js new file mode 100644 index 0000000..1b1f3cb --- /dev/null +++ b/src/core/rules/MissingContentForLayout.js @@ -0,0 +1,44 @@ +/** + * pos-supervisor:MissingContentForLayout rule — layouts missing + * `{{ content_for_layout }}`. Pre-rule the check landed as `.unmatched` + * even though fix-generator's `fixMissingContentForLayout` already inserts + * the placeholder after `` (or at line 0 if no body tag exists). + * + * The rule attaches stable attribution and a `guidance` fix that explains + * the relationship between `{{ content_for_layout }}` and named `{% yield %}` + * slots — the heuristic's `insert` text_edit is the actionable diff. + */ + +export const rules = [ + { + id: 'MissingContentForLayout.default', + check: 'pos-supervisor:MissingContentForLayout', + priority: 100, + when: () => true, + apply: () => ({ + rule_id: 'MissingContentForLayout.default', + hint_md: + 'Every layout MUST include `{{ content_for_layout }}` exactly once — that is where the page body ' + + 'is rendered into. Without it, pages using this layout serve only the layout chrome (header / nav / ' + + 'footer) and the page-specific content silently disappears.\n\n' + + 'Distinction:\n' + + ' • `{{ content_for_layout }}` — the implicit "page body" slot. Every layout has exactly one.\n' + + ' • `{% yield "name" %}` — named, optional slots a page can fill via `{% content_for "name" %}`. ' + + 'Use these for sidebars, head injection, etc. Adding more named slots does NOT replace the ' + + 'implicit body slot.', + fixes: [{ + type: 'guidance', + description: + 'Insert `{{ content_for_layout }}` once in the layout — typically right after the `` tag. ' + + 'The heuristic fix-generator emits the literal `insert` text_edit; accept it. ' + + 'Add named `{% yield "name" %}` slots only when pages need extra fill points.', + }], + confidence: 0.95, + see_also: { + tool: 'domain_guide', + args: { domain: 'layouts' }, + reason: 'Layouts domain guide explains content_for_layout vs yield and shows the canonical layout shape.', + }, + }), + }, +]; diff --git a/src/core/rules/MissingPage.js b/src/core/rules/MissingPage.js new file mode 100644 index 0000000..ad5e199 --- /dev/null +++ b/src/core/rules/MissingPage.js @@ -0,0 +1,173 @@ +/** + * MissingPage rule — `link_to '/foo'`, `redirect_to '/foo'`, etc. references + * a route the project doesn't serve. The diagnostic-pipeline already + * suppresses references whose page is on disk (via `buildPageRouteIndex` + + * `resolvePageRoute`); by the time the diagnostic reaches this rule the + * route is genuinely missing OR served with a different method. + * + * Pre-rule the check landed as `.unmatched`. The bare LSP message + * (`No page found for route '/foo' (GET)`) gives the agent no signal on + * whether to fix the route, change method, or create the page. + * + * Subrules: + * 10 — typo: extracted route is Levenshtein-close to an indexed page slug + * → suggest renaming the reference. + * 100 — default: emit a structured decision tree (typo / new page / method + * mismatch) and propose a `create_file` for the most-likely page path. + * + * Note: route ↔ method mismatch detection lives in the pipeline upstream. + * This rule treats every surviving diagnostic as a "page truly missing" + * outcome and points the agent at the three valid resolutions. + */ + +import { nearestByLevenshtein } from './queries.js'; + +export const rules = [ + { + id: 'MissingPage.typo', + check: 'MissingPage', + priority: 10, + when: (diag, facts) => { + const parsed = parseMissingPageMessage(diag.message); + if (!parsed) return false; + const candidates = pageRouteCandidates(facts?.graph); + if (candidates.length === 0) return false; + return nearestByLevenshtein(parsed.route, candidates, 3).length > 0; + }, + apply: (diag, facts) => { + const { route, method } = parseMissingPageMessage(diag.message); + const candidates = pageRouteCandidates(facts.graph); + const nearest = nearestByLevenshtein(route, candidates, 3); + const best = nearest[0]; + if (!best || best.distance > 3) return null; + const list = nearest.map(n => `\`/${n.name}\``).join(', '); + return { + rule_id: 'MissingPage.typo', + hint_md: + `No page serves \`/${route}\` (${method.toUpperCase()}), but the project has nearby routes: ${list}. ` + + `If the reference is a typo, fix it; if the page is genuinely missing, scaffold it now.`, + fixes: [{ + type: 'guidance', + description: + `Replace \`'/${route}'\` with \`'/${best.name}'\` in the link/redirect (or correct the slug ` + + `to match \`/${route}\` if you actually meant the latter). Distance ${best.distance} — ` + + `verify the correction before applying.`, + }], + confidence: best.distance <= 1 ? 0.85 : 0.7, + }; + }, + }, + + { + id: 'MissingPage.default', + check: 'MissingPage', + priority: 100, + when: () => true, + apply: (diag) => { + const parsed = parseMissingPageMessage(diag.message); + const route = parsed ? parsed.route : null; // can legitimately be '' for root + const method = parsed?.method ?? 'get'; + const haveRoute = route !== null; + const inferredPath = haveRoute ? routeToPagePath(route) : 'app/views/pages/.liquid'; + const routeSpan = haveRoute ? `\`/${route}\`` : 'this route'; + // The root page conventionally has either an empty `slug:` or none at + // all (the file-path → route fallback covers it). Distinguish the + // wording so an empty `slug:` line doesn't read like a typo. + const slugBlurb = !haveRoute + ? '(set `slug:` to the desired URL)' + : route === '' + ? '(omit `slug:` — the root page lives at `app/views/pages/index.liquid` and serves `/` automatically)' + : `(\`slug: ${route}\`)`; + + const hint = + `${routeSpan} (${method.toUpperCase()}) is not served by any page in this project. ` + + `Three valid resolutions:\n` + + ` • **Typo in the reference** — fix the slug at the call site (\`link_to\`, \`redirect_to\`, ` + + `\`form action\`, etc.).\n` + + ` • **New page** — scaffold a page at \`${inferredPath}\` ${slugBlurb}. ` + + `The file path alone determines the route when no \`slug:\` front-matter key is present.\n` + + ` • **Method mismatch** — a page may serve this URL for a different HTTP method (e.g. agent ` + + `wrote ${method.toUpperCase()} but the page is GET-only). Open the candidate page and check ` + + `its \`method:\` front-matter key.\n\n` + + `If you're mid-feature and the page is in the plan but not yet on disk, pass ` + + `\`pending_pages=["${inferredPath}"]\` to validate_code so this stops firing while you write it.`; + + return { + rule_id: 'MissingPage.default', + hint_md: hint, + fixes: [{ + type: 'create_file', + path: inferredPath, + description: + `Create the missing page at \`${inferredPath}\` (slug: \`${route ?? ''}\`). ` + + `Only apply if you intend to add this page — if the route was a typo at the call site, ` + + `fix the reference instead.`, + }], + confidence: 0.6, + see_also: { + tool: 'domain_guide', + args: { domain: 'pages' }, + reason: 'Pages domain guide explains slug/method semantics and the file-path → route mapping.', + }, + }; + }, + }, +]; + +/** + * Mirror of `parseMissingPageMessage` from page-route-index.js — kept local + * to avoid creating a load-order dependency between the rule engine and the + * pipeline. The shape matches; behaviour is identical for the messages we + * receive at this stage. Returns null when the message can't be parsed. + */ +function parseMissingPageMessage(message) { + if (!message) return null; + const quoted = message.match(/['"`]([^'"`]+)['"`]/); + if (!quoted) return null; + let route = quoted[1].trim(); + while (route.startsWith('/')) route = route.slice(1); + if (route === 'index') route = ''; + if (route.endsWith('/index')) route = route.slice(0, -'/index'.length); + const methodMatch = message.match(/\(([A-Za-z]+)\)/); + const method = (methodMatch?.[1] ?? 'get').toLowerCase(); + return { route, method }; +} + +/** + * Enumerate every page slug the project graph knows. Prefers the explicit + * front-matter slug when present; falls back to deriving from the file path + * exactly the way the route index does. + */ +function pageRouteCandidates(graph) { + if (!graph) return []; + const out = []; + for (const node of graph.nodesByType('page')) { + if (typeof node.slug === 'string' && node.slug.length > 0) { + out.push(normalize(node.slug)); + } else if (node.path) { + out.push(routeFromPath(node.path)); + } + } + return [...new Set(out)]; +} + +function normalize(raw) { + let p = raw.trim(); + while (p.startsWith('/')) p = p.slice(1); + if (p === 'index') return ''; + if (p.endsWith('/index')) p = p.slice(0, -'/index'.length); + return p; +} + +function routeFromPath(absLikePath) { + const stripped = absLikePath + .replace(/^app\/views\/pages\//, '') + .replace(/\.html\.liquid$/, '') + .replace(/\.liquid$/, ''); + return normalize(stripped); +} + +function routeToPagePath(route) { + if (!route || route === '') return 'app/views/pages/index.liquid'; + return `app/views/pages/${route}.liquid`; +} diff --git a/src/core/rules/MissingPartial.js b/src/core/rules/MissingPartial.js new file mode 100644 index 0000000..09c8df9 --- /dev/null +++ b/src/core/rules/MissingPartial.js @@ -0,0 +1,382 @@ +/** + * MissingPartial rules — first check fully ported to rule engine. + * + * Priority order: + * 5 — invalid_lib_prefix: literal `lib/commands/` or `lib/queries/` prefix → text_edit + * 10 — module_path: module partials → guidance + module_info see_also + * 20 — file_exists: target exists on disk but LSP still flags → guidance + * 30 — suggest_nearest: did-you-mean via Levenshtein on reachable partials + * 40 — create_file: generate create_file fix with scaffold + * 1000 — default: catch-all that fires when none of the above guards matched + * (missing extractor params, unrecognised path shape, etc.). Without + * this rule the diagnostic would land as `MissingPartial.unmatched` and + * the agent would see the bare LSP message — every `.unmatched` row in + * the dashboard analytics. The default is intentionally guidance-only, + * confidence 0.5, so it never preempts a more specific rule. + */ +import { classifyPath, nearestByLevenshtein, partialNames, partialsReachableFrom } from './queries.js'; +import { + installedModules, + moduleCallPathsByCategory, + moduleInstalled, +} from './module-paths.js'; + +export const rules = [ + { + // `function` tag paths resolve relative to the partial search paths + // (`app/views/partials/`, `app/lib/`), not project root, so `lib/commands/X` + // expands to `app/lib/lib/commands/X` which never exists. Drop the prefix + // — `commands/X` and `queries/X` are the canonical forms. + id: 'MissingPartial.invalid_lib_prefix', + check: 'MissingPartial', + priority: 5, + when: (diag) => { + const name = diag.params?.partial; + return !!name && (name.startsWith('lib/commands/') || name.startsWith('lib/queries/')); + }, + apply: (diag) => { + const name = diag.params.partial; + const corrected = name.slice('lib/'.length); + const category = name.startsWith('lib/commands/') ? 'command' : 'query'; + const hint = + `Drop the invalid \`lib/\` prefix from \`${name}\`. ` + + `\`function\` tag paths resolve from the partial search paths ` + + `(\`app/views/partials/\`, \`app/lib/\`) — a literal \`lib/\` prefix expands ` + + `to \`app/lib/lib/${corrected}\` which never exists. ` + + `Use \`${corrected}\` instead.`; + + const fix = buildLibPrefixTextEdit(diag, name, corrected) ?? { + type: 'guidance', + description: + `Drop the \`lib/\` prefix from the ${category} call: replace \`${name}\` with \`${corrected}\` ` + + `in the \`{% function %}\` tag on line ${diag.line ?? '?'}.`, + }; + + return { + rule_id: 'MissingPartial.invalid_lib_prefix', + hint_md: hint, + fixes: [fix], + confidence: 0.95, + }; + }, + }, + + { + id: 'MissingPartial.module_path', + check: 'MissingPartial', + priority: 10, + when: (diag) => { + const name = diag.params?.partial; + return !!name && name.startsWith('modules/'); + }, + apply: (diag, facts) => { + const name = diag.params.partial; + const parsed = parseModulePath(name); + const projectDir = facts?.projectDir ?? null; + + // 1. Module not installed → list known modules + Levenshtein. + if (parsed.moduleName && projectDir && !moduleInstalled(projectDir, parsed.moduleName)) { + const installed = installedModules(projectDir); + const nearest = nearestByLevenshtein(parsed.moduleName, installed, 3); + const list = installed.length > 0 + ? `Installed modules: ${installed.map(m => `\`${m}\``).join(', ')}.` + : `No modules are installed under \`modules/\`.`; + const didYouMean = nearest.length > 0 + ? ` Did you mean \`${nearest[0].name}\`?` + : ''; + return { + rule_id: 'MissingPartial.module_path', + hint_md: + `Module \`${parsed.moduleName}\` is not installed in this project. ${list}${didYouMean} ` + + `Module paths look like \`modules///\` — check the module name first.`, + fixes: [{ + type: 'guidance', + description: + `Module \`${parsed.moduleName}\` not installed. Either install it (\`pos-cli modules install ${parsed.moduleName}\`), ` + + `pick a different module from the installed list, or move the call into a project-local file under \`app/lib/\`.`, + }], + confidence: 0.9, + see_also: { + tool: 'project_map', + args: {}, + reason: `Module '${parsed.moduleName}' not installed. project_map enumerates installed modules and project-local commands/queries.`, + }, + }; + } + + // 2. Module installed → enumerate exports and reason about the bad path. + const moduleName = parsed.moduleName; + const exportsByCategory = projectDir && moduleName + ? moduleCallPathsByCategory(projectDir, moduleName) + : null; + + const buildCheckSpecial = parsed.category === 'commands' + && (parsed.rest === 'build' || parsed.rest === 'check'); + + const allExports = exportsByCategory + ? Object.values(exportsByCategory).flat() + : []; + + // Levenshtein over every callable in the module, not just the + // (possibly mistyped) category — agents land in the wrong category + // bucket all the time (e.g. `commands/find_user` when the export is + // `queries/users/find`). + const nearest = allExports.length > 0 + ? nearestByLevenshtein(name, allExports, 5) + : []; + + const candidatesBlock = nearest.length > 0 + ? nearest.map(n => `\`${n.name}\``).join(', ') + : '(no close matches in this module)'; + + let lead; + if (buildCheckSpecial) { + // The original failure mode the report flagged: agents copy the + // `modules/core/commands/execute` shortcut, then assume `…/build` + // and `…/check` exist as siblings. They don't — build/check are + // inline phases of the *caller's* command, written next to + // execute.liquid in the agent's own `app/lib/commands//` + // tree. Only `execute` is exported by core for the simple-create + // shortcut. + lead = + `\`${name}\` does not exist. \`build\` and \`check\` are **inline phases of your own command**, ` + + `not module-level helpers — write them as \`build.liquid\` / \`check.liquid\` next to your \`execute.liquid\` ` + + `under \`app/lib/commands//\`. Only \`modules/${moduleName}/commands/execute\` is exported by core ` + + `(simple-create shortcut). For complex flows (multi-step orchestration, validation chains) ` + + `keep build/check inline.`; + } else { + lead = `\`${name}\` is not exported by module \`${moduleName}\`.`; + } + + const categorySummary = exportsByCategory + ? Object.entries(exportsByCategory) + .filter(([, paths]) => paths.length > 0) + .map(([cat, paths]) => `${cat} (${paths.length})`) + .join(', ') + : null; + + const hint = + `${lead}\n\n` + + `Closest matches in \`${moduleName}\`: ${candidatesBlock}.` + + (categorySummary ? `\nExported categories: ${categorySummary}.` : '') + + `\nCall \`module_info(${moduleName}, api)\` to read the full export list with @param signatures.`; + + const fixDescription = buildCheckSpecial + ? `Remove the \`{% function ... = '${name}', ... %}\` call and inline the build/check logic ` + + `directly in this file (or its sibling phase file). If you intended a different module helper, ` + + `replace the path with one of: ${nearest.slice(0, 3).map(n => `\`${n.name}\``).join(', ') || '(none)'}. ` + + `Use \`module_info(${moduleName}, api)\` for the full list.` + : `Replace \`${name}\` with the closest valid export: ${nearest.slice(0, 3).map(n => `\`${n.name}\``).join(', ') || '(none)'}, ` + + `or call \`module_info(${moduleName}, api)\` to see every callable path the module exposes.`; + + return { + rule_id: 'MissingPartial.module_path', + hint_md: hint, + fixes: [{ + type: 'guidance', + description: fixDescription, + }], + confidence: nearest.length > 0 ? 0.9 : 0.7, + see_also: { + tool: 'module_info', + args: { name: moduleName, section: 'api' }, + reason: `module_info(${moduleName}, api) returns live-scanned call paths and @param signatures for every export.`, + }, + }; + }, + }, + + { + id: 'MissingPartial.file_exists', + check: 'MissingPartial', + priority: 20, + when: (diag, facts) => { + const name = diag.params?.partial; + if (!name) return false; + const { path } = classifyPath(name); + return path && facts.graph.hasNode(path); + }, + apply: (diag) => { + const { path } = classifyPath(diag.params.partial); + return { + rule_id: 'MissingPartial.file_exists', + hint_md: `File \`${path}\` exists but the linter still reports it as missing. Check that the file is not empty, has no syntax errors, and the path in the render/function tag matches exactly.`, + fixes: [{ + type: 'guidance', + description: `File \`${path}\` exists on disk. Verify: (1) file is not empty, (2) no Liquid syntax errors inside it, (3) the render/function tag path matches exactly (case-sensitive).`, + }], + confidence: 0.7, + }; + }, + }, + + { + id: 'MissingPartial.suggest_nearest', + check: 'MissingPartial', + priority: 30, + when: (diag, facts) => { + const name = diag.params?.partial; + if (!name || name.startsWith('modules/')) return false; + const { type } = classifyPath(name); + if (type === 'module') return false; + const candidates = type === 'partial' + ? partialNames(facts.graph) + : []; // commands/queries use exact paths + return candidates.length > 0; + }, + apply: (diag, facts) => { + const name = diag.params.partial; + const { type, path: targetPath } = classifyPath(name); + + let candidates; + if (type === 'partial') { + // Prefer partials reachable from the caller's dependency tree + const reachable = diag.file ? partialsReachableFrom(facts.graph, diag.file) : []; + candidates = reachable.length > 0 ? reachable : partialNames(facts.graph); + } else { + candidates = []; + } + + const nearest = nearestByLevenshtein(name, candidates, 5); + if (nearest.length === 0) return null; + + const suggestions = nearest.map(n => `\`${n.name}\` (distance: ${n.distance})`).join(', '); + const tag = type === 'partial' ? 'render' : 'function'; + + const bestMatch = nearest[0].name; + return { + rule_id: 'MissingPartial.suggest_nearest', + hint_md: `\`${name}\` not found. Did you mean: ${suggestions}? Fix the name in the \`{% ${tag} %}\` tag.`, + fixes: [{ + type: 'guidance', + description: `Replace \`${name}\` with \`${bestMatch}\` in the \`{% ${tag} '${name}' %}\` tag.`, + }], + confidence: 0.6, + }; + }, + }, + + { + id: 'MissingPartial.create_file', + check: 'MissingPartial', + priority: 40, + when: (diag) => { + const name = diag.params?.partial; + if (!name || name.startsWith('modules/')) return false; + const { path } = classifyPath(name); + return !!path; + }, + apply: (diag, facts) => { + const name = diag.params.partial; + const { type, path: targetPath } = classifyPath(name); + + // Constraint: path must not collide with existing node + if (facts.graph.hasNode(targetPath)) return null; + + // Constraint: path follows convention + if (type === 'partial' && !targetPath.startsWith('app/views/partials/')) return null; + if (type === 'command' && !targetPath.startsWith('app/lib/commands/')) return null; + if (type === 'query' && !targetPath.startsWith('app/lib/queries/')) return null; + + const tag = type === 'partial' ? 'render' : 'function'; + + return { + rule_id: 'MissingPartial.create_file', + hint_md: `Create missing file: \`${targetPath}\`. Use \`scaffold\` tool or create manually with appropriate \`{% doc %}\` block.`, + fixes: [{ + type: 'create_file', + path: targetPath, + description: `Create missing ${type}: \`${targetPath}\``, + }], + confidence: 0.8, + }; + }, + }, + + // Last-resort catch-all. Fires when none of the specialised guards above + // matched — typically because the LSP message did not parse into a + // `params.partial` (an upstream message-shape change), or because the path + // shape (`type` from classifyPath) did not fit any existing rule. Hint + // surfaces the three canonical resolutions with no false specifics, so the + // agent gets actionable guidance instead of `.unmatched` + bare LSP text. + { + id: 'MissingPartial.default', + check: 'MissingPartial', + priority: 1000, + when: () => true, + apply: (diag) => { + const name = diag.params?.partial ?? null; + const ref = name ? `\`${name}\`` : 'this reference'; + return { + rule_id: 'MissingPartial.default', + hint_md: + `${ref} does not resolve to any partial, command, or query in the project. ` + + `Three canonical resolutions:\n` + + ` • **Typo** — fix the path in the \`{% render %}\` / \`{% function %}\` tag.\n` + + ` • **Missing file** — create the target. Partials live under \`app/views/partials/\`, ` + + `commands under \`app/lib/commands/\`, queries under \`app/lib/queries/\`.\n` + + ` • **Wrong prefix** — \`function\` paths resolve from \`app/lib/\`, so \`lib/commands/X\` ` + + `expands to \`app/lib/lib/commands/X\` and never resolves. Drop the leading \`lib/\`.\n\n` + + `Run \`project_map\` to enumerate the partials, commands, and queries this project actually has.`, + fixes: [{ + type: 'guidance', + description: name + ? `Verify the path \`${name}\` against \`project_map\` output, then either correct the typo, drop a leading \`lib/\` if present, or create the file at the canonical location.` + : `Run \`project_map\` to enumerate available partials, commands, and queries; reconcile the failing reference against the live list.`, + }], + confidence: 0.5, + see_also: { + tool: 'project_map', + args: {}, + reason: 'project_map lists every partial, command, and query the project serves — the authoritative source for resolving a missing reference.', + }, + }; + }, + }, +]; + +/** + * Build a `text_edit` fix that swaps a quoted partial reference for its + * `lib/`-stripped form. Returns null when the diagnostic lacks the position + * fields LSP normally provides (line/column/endColumn) — callers fall back + * to a guidance fix in that case. + * + * The replacement quotes with `'` (single-quote convention used throughout + * platformOS templates and our scaffolds). The rule engine has no access to + * the source buffer, so a perfect echo of the user's quote style can't be + * preserved here; `fix-generator.js` carries content and re-emits the fix + * with the correct quote when a buffer is available. + */ +function buildLibPrefixTextEdit(diag, name, corrected) { + if (diag.line == null || diag.column == null || diag.endColumn == null) return null; + return { + type: 'text_edit', + range: { + start: { line: diag.line, character: diag.column }, + end: { line: diag.endLine ?? diag.line, character: diag.endColumn }, + }, + new_text: `'${corrected}'`, + description: + `Drop invalid \`lib/\` prefix — function paths resolve from \`app/lib/\`. ` + + `Replace \`${name}\` with \`${corrected}\`.`, + }; +} + +/** + * Split `modules///` into its parts. The returned + * `category` is the literal first segment after the module name (callers + * decide whether it maps to a known module-export bucket); `rest` is the + * remainder joined with '/'. Returns nulls when the input doesn't fit the + * shape so callers can shortcut. + */ +export function parseModulePath(name) { + if (!name || !name.startsWith('modules/')) { + return { moduleName: null, category: null, rest: null }; + } + const parts = name.split('/'); + // parts[0] === 'modules' + const moduleName = parts[1] ?? null; + const category = parts[2] ?? null; + const rest = parts.length > 3 ? parts.slice(3).join('/') : null; + return { moduleName, category, rest }; +} diff --git a/src/core/rules/MissingRenderPartialArguments.js b/src/core/rules/MissingRenderPartialArguments.js new file mode 100644 index 0000000..0838a9d --- /dev/null +++ b/src/core/rules/MissingRenderPartialArguments.js @@ -0,0 +1,84 @@ +/** + * MissingRenderPartialArguments rules — rich cross-file hints for missing params. + * + * Priority order: + * 10 — doc_block_mismatch: reads target partial's params, shows full expected signature + * 20 — chain_satisfied: param available in caller's own scope (received from grandparent) + * 30 — optional_param: param has a default → downgrade to info-level confidence + * 100 — generic: standard hint + */ +import { getPartialParams, isParamAvailableInCallerScope } from '../render-flow.js'; + +export const rules = [ + { + id: 'MissingRenderPartialArguments.doc_block_mismatch', + check: 'MissingRenderPartialArguments', + priority: 10, + when: (diag, facts) => { + const partialName = diag.params?.partial; + if (!partialName) return false; + const params = getPartialParams(facts.graph, partialName); + return params.length > 0; + }, + apply: (diag, facts) => { + const partialName = diag.params.partial; + const missingParam = diag.params.missing_param ?? 'unknown'; + const params = getPartialParams(facts.graph, partialName); + const signature = params.map(p => `${p}: ${p}`).join(', '); + + return { + rule_id: 'MissingRenderPartialArguments.doc_block_mismatch', + hint_md: `Required param \`${missingParam}\` is not passed to \`${partialName}\`.\n\nFull signature: \`{% render '${partialName}', ${signature} %}\`\n\nDeclared params: ${params.map(p => `\`${p}\``).join(', ')}. Add the missing argument to the render/function call.`, + suggestion: `Add \`, ${missingParam}: ${missingParam}\` to the render call.`, + fixes: [], + confidence: 0.9, + see_also: { + tool: 'domain_guide', + args: { domain: 'partials', section: 'api' }, + reason: `Render call missing required param. domain_guide(partials, api) explains {% doc %} @param declarations.`, + }, + }; + }, + }, + + { + id: 'MissingRenderPartialArguments.chain_satisfied', + check: 'MissingRenderPartialArguments', + priority: 20, + when: (diag, facts) => { + const missingParam = diag.params?.missing_param; + if (!missingParam || !diag.file) return false; + return isParamAvailableInCallerScope(facts.graph, diag.file, missingParam); + }, + apply: (diag) => { + const partialName = diag.params.partial ?? 'unknown'; + const missingParam = diag.params.missing_param; + + return { + rule_id: 'MissingRenderPartialArguments.chain_satisfied', + hint_md: `Param \`${missingParam}\` is not passed to \`${partialName}\`, but this file declares \`${missingParam}\` as its own param (received from a caller). Add \`${missingParam}: ${missingParam}\` to forward it.`, + suggestion: `Forward the param: add \`, ${missingParam}: ${missingParam}\` to the render call.`, + fixes: [], + confidence: 0.85, + }; + }, + }, + + { + id: 'MissingRenderPartialArguments.generic', + check: 'MissingRenderPartialArguments', + priority: 100, + when: () => true, + apply: (diag) => { + const partialName = diag.params?.partial ?? 'unknown'; + const missingParam = diag.params?.missing_param ?? 'unknown'; + + return { + rule_id: 'MissingRenderPartialArguments.generic', + hint_md: `Required param \`${missingParam}\` is not passed to \`${partialName}\`. Open the partial's \`{% doc %}\` block to see the full signature, then add the missing argument to the render/function call.`, + fixes: [], + confidence: 0.5, + }; + }, + }, +]; diff --git a/src/core/rules/MissingSlug.js b/src/core/rules/MissingSlug.js new file mode 100644 index 0000000..58d7188 --- /dev/null +++ b/src/core/rules/MissingSlug.js @@ -0,0 +1,42 @@ +/** + * pos-supervisor:MissingSlug rule — pages without a `slug:` in their front + * matter. Pre-rule the check landed as `.unmatched` even though the + * fix-generator already produces an `insert` text_edit + * (`fixMissingSlugInsert`) that prefills a sensible slug from the file + * path. This rule promotes the check to a stable rule_id and ships a + * `guidance` fix that explains *why* the slug matters — the heuristic's + * literal text_edit remains the actionable diff. + */ + +export const rules = [ + { + id: 'MissingSlug.default', + check: 'pos-supervisor:MissingSlug', + priority: 100, + when: () => true, + apply: () => ({ + rule_id: 'MissingSlug.default', + hint_md: + 'Page is missing `slug:` in its front matter. Without an explicit slug the platform falls back to ' + + 'a path derived from the filename — fine for one-off pages, but unstable when the file is renamed ' + + 'or moved. Set `slug:` explicitly so URLs are owned by the page, not the filesystem.\n\n' + + 'Conventions:\n' + + ' • Use kebab-case and avoid the file extension (`slug: contact`, not `slug: contact.liquid`).\n' + + ' • Use `:param` for dynamic segments (`slug: posts/:id`), not `[param]` (Next.js style).\n' + + ' • No leading slash — `slug: foo`, not `slug: /foo`.', + fixes: [{ + type: 'guidance', + description: + 'Add `slug: ` between the front matter `---` markers. ' + + 'The heuristic fix-generator proposes a slug derived from the filename — accept it ' + + 'unless the public URL should diverge from the path.', + }], + confidence: 0.85, + see_also: { + tool: 'domain_guide', + args: { domain: 'pages' }, + reason: 'Pages domain guide describes slug conventions and how dynamic segments resolve.', + }, + }), + }, +]; diff --git a/src/core/rules/NonGetRenderingPage.js b/src/core/rules/NonGetRenderingPage.js new file mode 100644 index 0000000..f79f1b1 --- /dev/null +++ b/src/core/rules/NonGetRenderingPage.js @@ -0,0 +1,183 @@ +/** + * NonGetRenderingPage rules — three distinct misconfigurations of page + * `method:` / `format:` / form `action` per the 2026-04-27 gist analysis + * (NonGetRenderingPageRule.md). The structural emitter + * (`validatePageMethodAndForms` in structural-warnings.js) is the only + * producer of `pos-supervisor:NonGetRenderingPage` and tags each emit with + * a leading-clause discriminator the rule layer routes by. + * + * Subrule analytics IDs: + * • NonGetRenderingPage.api_renders_html — API-pathed page (`/api/`, + * `/_/`, `/internal/`) is non-GET but emits HTML or omits `format: json`. + * • NonGetRenderingPage.html_on_post — non-API page is non-GET but + * renders HTML; browser GETs return 404. + * • NonGetRenderingPage.get_form_target — GET page hosts a + * `` whose action is not under an + * internal-API prefix and is not the page's own slug. + * • NonGetRenderingPage.default — fallback for diagnostics that + * don't match any subrule discriminator (defensive — should not fire + * in practice once the emitter is in sync with this router). + * + * Each subrule emits a concrete `guidance` fix that names the right shape + * to converge on. No `text_edit` here: the deterministic fix requires + * cross-file changes (edit page front matter AND add an API endpoint AND + * possibly rewrite the form attribute) which the rule layer can't compose + * safely. Agents accept the guidance, then validate iteratively. + */ + +const API_RENDERS_HTML_RE = /^API page \(slug `([^`]+)`\) has `method: (\w+)`/; +const HTML_ON_POST_RE = /^Page has `method: (\w+)` but renders HTML/; +const GET_FORM_TARGET_RE = /^Form on GET page posts to `([^`]+)`/; + +export const rules = [ + { + id: 'NonGetRenderingPage.api_renders_html', + check: 'pos-supervisor:NonGetRenderingPage', + priority: 5, + when: (diag) => API_RENDERS_HTML_RE.test(diag.message ?? ''), + apply: (diag) => { + const m = diag.message.match(API_RENDERS_HTML_RE); + const slug = m?.[1] ?? ''; + const method = m?.[2] ?? 'post'; + return { + rule_id: 'NonGetRenderingPage.api_renders_html', + hint_md: + `API page \`${slug}\` is set to \`method: ${method}\` but is configured to render HTML — ` + + `either it carries a \`layout:\` / inline HTML, or it is missing \`format: json\`. ` + + `Pages under \`/api/\`, \`/_/\`, or \`/internal/\` must respond with JSON; rendering HTML to ` + + `a JSON-expecting client is a silent contract break.\n\n` + + `Canonical shape:\n` + + '```liquid\n' + + `---\n` + + `slug: ${slug.replace(/^\//, '')}\n` + + `method: ${method}\n` + + `format: json\n` + + `---\n` + + `{% graphql result = 'mutation_path', ...args %}\n` + + `{{ result | json }}\n` + + '```', + fixes: [{ + type: 'guidance', + description: + `Add \`format: json\` to the front matter, drop any \`layout:\` line, and replace the body ` + + `with a \`{% graphql %}\` call followed by \`{{ result | json }}\`. ` + + `Keep \`method: ${method}\` so the verb still matches the form / fetch caller.`, + }], + confidence: 0.9, + see_also: { + tool: 'domain_guide', + args: { domain: 'api-calls' }, + reason: 'API endpoint conventions in platformOS — JSON format, GraphQL bodies, no layout.', + }, + }; + }, + }, + + { + id: 'NonGetRenderingPage.html_on_post', + check: 'pos-supervisor:NonGetRenderingPage', + priority: 10, + when: (diag) => HTML_ON_POST_RE.test(diag.message ?? ''), + apply: (diag) => { + const m = diag.message.match(HTML_ON_POST_RE); + const method = m?.[1] ?? 'post'; + return { + rule_id: 'NonGetRenderingPage.html_on_post', + hint_md: + `Page renders HTML but is set to \`method: ${method}\`. Browsers always issue GET — ` + + `every navigation to this URL will 404. Two valid shapes:\n\n` + + `**Landing / display page** — drop the \`method:\` field (or set \`method: get\`); have any ` + + `embedded form POST to a separate API endpoint:\n` + + '```liquid\n' + + `---\nslug: contact\n---\n` + + `\n \n\n` + + '```\n' + + `**Form-handling endpoint** — rename the slug under \`/api/\` and switch to JSON output:\n` + + '```liquid\n' + + `---\nslug: api/contacts/create\nmethod: ${method}\nformat: json\n---\n` + + `{% graphql result = 'contacts/create', ...context.params.contact %}\n` + + `{{ result | json }}\n` + + '```', + fixes: [{ + type: 'guidance', + description: + `Decide intent: (a) **landing page** — remove \`method: ${method}\` from front matter; the ` + + `form on this page should action to an \`/api/...\` slug. (b) **API handler** — move the slug ` + + `under \`/api/\`, add \`format: json\`, replace the HTML body with a \`{% graphql %}\` call.`, + }], + confidence: 0.9, + see_also: { + tool: 'domain_guide', + args: { domain: 'pages' }, + reason: 'Page method semantics — GET serves browsers, non-GET handles form / fetch payloads.', + }, + }; + }, + }, + + { + id: 'NonGetRenderingPage.get_form_target', + check: 'pos-supervisor:NonGetRenderingPage', + priority: 15, + when: (diag) => GET_FORM_TARGET_RE.test(diag.message ?? ''), + apply: (diag) => { + const m = diag.message.match(GET_FORM_TARGET_RE); + const action = m?.[1] ?? ''; + const stripped = action.replace(/^\/+/, ''); + const apiAction = `/api/${stripped}`; + const apiPagePath = `app/views/pages/api/${stripped}.liquid`; + return { + rule_id: 'NonGetRenderingPage.get_form_target', + hint_md: + `\`
\` on this GET page posts to \`${action}\`. That action target is not under an ` + + `internal-API prefix (\`/api/\`, \`/_/\`, \`/internal/\`) and isn't the page's own slug, ` + + `so the submission has nowhere valid to land — unless an explicit \`method: post\` page already ` + + `serves \`${action}\`. The canonical fix is to route the form through an API page:\n\n` + + `1. Update the form action: \`\`.\n` + + `2. Create the API page at \`${apiPagePath}\` with \`method: post\`, \`format: json\`, and a ` + + `\`{% graphql %}\` body.`, + fixes: [{ + type: 'guidance', + description: + `Change the form action from \`${action}\` to \`${apiAction}\` and create ` + + `\`${apiPagePath}\` as a \`method: post\` / \`format: json\` page. ` + + `Alternative (only if you control \`${action}\` already): verify that \`${action}\` is served ` + + `by a page with \`method: post\` — if not, NonGetRenderingPage.html_on_post will fire there.`, + }], + confidence: 0.85, + see_also: { + tool: 'domain_guide', + args: { domain: 'forms' }, + reason: 'Form submission patterns — actions must hit a page with the matching `method:` verb.', + }, + }; + }, + }, + + { + id: 'NonGetRenderingPage.default', + check: 'pos-supervisor:NonGetRenderingPage', + priority: 100, + when: () => true, + apply: (diag) => ({ + rule_id: 'NonGetRenderingPage.default', + hint_md: + `Page method or form-target configuration is off. Read the upstream message for specifics — ` + + `the canonical platformOS shapes are:\n` + + ` • UI page → \`method: get\` (or omit), HTML body, layout allowed.\n` + + ` • API endpoint → slug under \`/api/\`, \`method: post\`/etc., \`format: json\`, no layout, ` + + `body is \`{% graphql %}\` + \`{{ result | json }}\`.\n` + + ` • Forms on GET pages → \`action="/api/"\` so the POST lands on the API page.`, + fixes: [{ + type: 'guidance', + description: + `Decide whether this page is a UI page (GET, HTML) or an API page (non-GET, JSON). ` + + `Convert front matter and body to match — see \`domain_guide(pages)\` for the canonical layouts.`, + }], + confidence: 0.6, + }), + }, +]; + +// Re-exported for tests + diagnostic-pipeline introspection. +export const _internal = { API_RENDERS_HTML_RE, HTML_ON_POST_RE, GET_FORM_TARGET_RE }; diff --git a/src/core/rules/OrphanedPartial.js b/src/core/rules/OrphanedPartial.js new file mode 100644 index 0000000..b16ccde --- /dev/null +++ b/src/core/rules/OrphanedPartial.js @@ -0,0 +1,120 @@ +/** + * OrphanedPartial rule — partial files that aren't rendered by anything + * else in the project graph. + * + * Pre-rule the check landed as `.unmatched`. The diagnostic-pipeline already + * suppresses commands/queries (invoked via `function`, not `render`) and + * pending-plan files via `suppressOrphanedPartial` upstream of this rule, so + * by the time we see one it's a real orphan in `app/views/partials/`. + * + * Two distinct intents → two recommendations: + * • The file is dead code → delete it (high-confidence delete_file fix when + * `referencedBy(file)` is empty in the graph). + * • The file is mid-development and the caller hasn't been written yet → + * pass `pending_files=[…callerPath]` to validate_code, or just write the + * caller now. The hint surfaces this option so agents don't blindly + * delete in-progress work. + * + * No fix when `diag.file` is missing — without the path we can't tell which + * file to act on. + */ + +import { dependentsOf, classifyFileType } from './queries.js'; + +export const rules = [ + { + id: 'OrphanedPartial.default', + check: 'OrphanedPartial', + priority: 100, + when: () => true, + apply: (diag, facts) => { + const file = diag.file ?? null; + const fileType = file ? classifyFileType(file) : 'unknown'; + const callers = file && facts?.graph ? dependentsOf(facts.graph, file) : []; + const callerCount = callers.length; + const filenameSpan = file ? `\`${file}\`` : 'this partial'; + + // The diagnostic-pipeline's `suppressOrphanedPartial` already drops + // commands/queries — by the time we see one, it should be a real + // partial. Belt-and-suspenders: if a non-partial slips through, give + // a softer "verify caller graph" message instead of a delete proposal. + const isPartial = fileType === 'partial'; + const isLayout = fileType === 'layout'; + + const fixes = []; + if (isPartial && callerCount === 0) { + fixes.push({ + type: 'delete_file', + path: file, + description: + `Delete ${filenameSpan} — no other file in the project renders it. ` + + `Re-run validate_code first if you're mid-feature; pass ` + + `\`pending_files=["${file}"]\` (or the caller's path) to suppress this warning while you write the caller.`, + }); + } + fixes.push({ + type: 'guidance', + description: orphanGuidance(filenameSpan, isPartial, isLayout), + }); + + return { + rule_id: 'OrphanedPartial.default', + hint_md: orphanHint(filenameSpan, callerCount, isPartial, isLayout), + fixes, + confidence: isPartial && callerCount === 0 ? 0.85 : 0.6, + }; + }, + }, +]; + +function orphanHint(filenameSpan, callerCount, isPartial, isLayout) { + if (isLayout) { + return ( + `${filenameSpan} is a layout with no pages selecting it via \`layout: \`. ` + + `Either select it from a page (\`---\\nlayout: ${filenameSpan.replace(/`/g, '').replace(/.*\//, '').replace(/\.liquid$/, '')}\\n---\`) ` + + `or delete the layout if it's no longer needed. Pages without a \`layout:\` key ` + + `default to \`application.liquid\`.` + ); + } + if (!isPartial) { + return ( + `${filenameSpan} appears to be unreferenced, but the file isn't a regular partial — ` + + `verify caller graph manually with \`platformos_references\` before deleting. ` + + `Commands and queries are invoked via \`{% function %}\` and may not always show up as ` + + `dependencies in the rendering graph.` + ); + } + const callerNote = callerCount === 0 + ? 'No file in the project renders or includes it.' + : `Found ${callerCount} caller(s) outside the standard render graph — verify with \`platformos_references\`.`; + return ( + `${filenameSpan} is an orphaned partial. ${callerNote}\n\n` + + `Two valid resolutions:\n` + + ` • **Dead code** — delete the file. Use the \`delete_file\` fix if you're certain nothing renders it.\n` + + ` • **Work in progress** — the caller hasn't been written yet. Either write the caller now ` + + `(then re-validate, the warning clears), or pass \`pending_files=[""]\` to ` + + `validate_code so the orphan is suppressed during the multi-file plan.\n\n` + + `Renaming the file (e.g. via scaffold output) is the third common cause — re-run ` + + `\`project_map\` to confirm callers point at the current name.` + ); +} + +function orphanGuidance(filenameSpan, isPartial, isLayout) { + if (isLayout) { + return ( + `Either set \`layout: \` in a page's front matter to use ${filenameSpan}, or delete the file ` + + `if it's no longer needed. Pages without an explicit \`layout:\` use \`application.liquid\`.` + ); + } + if (!isPartial) { + return ( + `Run \`platformos_references\` to enumerate every file that references ${filenameSpan}. ` + + `Commands/queries invoked via \`{% function %}\` may be missed by the render-graph orphan check.` + ); + } + return ( + `Decide: (a) delete ${filenameSpan} if it's dead, (b) write the calling page/partial if work is in progress, ` + + `or (c) pass \`pending_files=[""]\` to validate_code while you scaffold the caller. ` + + `Run \`platformos_references\` to confirm no rendering graph entry points at it.` + ); +} diff --git a/src/core/rules/ParserBlockingScript.js b/src/core/rules/ParserBlockingScript.js new file mode 100644 index 0000000..54d4a52 --- /dev/null +++ b/src/core/rules/ParserBlockingScript.js @@ -0,0 +1,46 @@ +/** + * ParserBlockingScript rule — `` placed at the very end of `` — the legacy workaround. ' + + 'Prefer `defer` for new code.\n\n' + + 'Inline scripts (`` with no `src`) are unaffected by this check.', + fixes: [{ + type: 'guidance', + description: + 'Add `defer` to the opening ` +Logo +``` + +### User Uploads -Before every change, verify: +Uploads are dynamic files stored per-record. -- [ ] No underscore prefix in partial filenames -- [ ] `render 'path/name'` maps to `app/views/partials/path/name.liquid` -- [ ] Pages have ONE HTTP method each -- [ ] No raw GraphQL in pages (use `{% graphql %}` tag with `.graphql` files) -- [ ] No HTML/JS/CSS in pages -- [ ] No hardcoded text in partials (use translations) -- [ ] `platformos-check` passes with 0 errors -- [ ] Every file synced after modification -- [ ] All list queries support pagination (`per_page`, `page`) -- [ ] All inputs validated in commands before persisting -- [ ] CSS/JS minified, `asset_url` used for cache busting +**Table Definition:** +```yaml +name: product +properties: + - name: image + type: upload +``` -### Asset URL Usage +**Form:** +```liquid +{% form %} + +{% endform %} +``` +**Displaying Uploads:** ```liquid -{{ 'images/img.png' | asset_url }} +{% graphql product = 'get_product', id: id %} +{{ product.record.properties.image.file_name }} +``` + +**Upload Properties:** + +| Property | Description | +|----------|-------------| +| `url` | Direct file URL | +| `file_name` | Original filename | +| `content_type` | MIME type | +| `size` | File size in bytes | + +### Assets vs Uploads + +| Aspect | Assets | Uploads | +|--------|--------|---------| +| Location | `app/assets/` | Record properties | +| Use Case | Static files (CSS, JS, logos) | Dynamic content | +| Quantity | Thousands expected | Millions supported | +| CDN | Yes | Yes | +| Max Size | 2GB | 2GB | + +### Direct S3 Upload + +platformOS uses **direct S3 upload** - files go straight to AWS S3 without passing through the application server. + +**Advantages:** +- **Speed** - No middleman, faster uploads +- **Cost** - Less bandwidth and server load +- **Security** - No file processing on app server +- **Scalability** - Handle unlimited concurrent uploads +- **Size** - Up to 5GB single file, 5TB multipart + +**Upload Flow:** +``` +1. User selects file +2. Browser requests signed S3 URL from platformOS +3. Browser uploads directly to S3 +4. S3 returns success +5. platformOS saves file reference to record +``` + +### Upload Configuration Options + +**Table Definition with Options:** +```yaml +name: product +properties: + - name: image + type: upload + options: + public: true # Public or private access + max_size: 5242880 # 5MB in bytes + versions: + - name: thumbnail + resize: '200x200>' # Resize to fit 200x200 + - name: medium + resize: '800x600>' + extensions: + - jpg + - png + - gif +``` + +### Upload Versions + +Automatically generate resized versions: + +```yaml +properties: + - name: photo + type: upload + options: + versions: + - name: thumb + resize: '100x100#' # Exact fit, may crop + - name: medium + resize: '300x300>' # Fit within, no upscale + - name: large + resize: '800x800>' ``` + +**Access versions in Liquid:** +```liquid +{{ product.properties.photo.url }} # Original +{{ product.properties.photo.versions.thumb.url }} # Thumbnail +{{ product.properties.photo.versions.medium.url }} # Medium +``` + +### Image Processing Options + +| Option | Description | Example | +|--------|-------------|---------| +| `resize: '100x100'` | Resize to dimensions | Fit within | +| `resize: '100x100>'` | Resize only if larger | Downscale only | +| `resize: '100x100<'` | Resize only if smaller | Upscale only | +| `resize: '100x100#'` | Exact dimensions | May crop | +| `resize: '100x100^'` | Minimum dimensions | May crop | + +--- + +## 16. Best Practices + +### Code Organization + +``` +app/ +├── views/ +│ ├── pages/ # Route handlers +│ ├── layouts/ # Page wrappers +│ └── partials/ +│ ├── components/ # UI components +│ ├── forms/ # Form partials +│ └── helpers/ # Utility partials +├── forms/ # Form configurations +├── graphql/ # Data queries +│ ├── records/ +│ ├── users/ +│ └── system/ +└── schema/ # Table definitions +``` + +### Naming Conventions + +| Component | Convention | Example | +|-----------|------------|---------| +| Tables | snake_case | `blog_post` | +| Properties | snake_case | `published_at` | +| Pages | snake_case | `about_us.liquid` | +| Partials | snake_case | `header.liquid` | +| Forms | snake_case | `contact_form.liquid` | +| GraphQL | snake_case | `get_blog_posts.graphql` | + +### Security Best Practices + +1. **Always use authorization policies** for protected routes +2. **Validate all inputs** using form validations +3. **Escape output** using Liquid's auto-escaping +4. **Use HTTPS** for all production instances +5. **Store secrets** in Partner Portal constants, not code +6. **Sanitize user content** before displaying + +### Performance Best Practices + +1. **Use pagination** for all list queries +2. **Load related records** in single GraphQL query +3. **Use background jobs** for long operations +4. **Cache expensive queries** using static cache +5. **Optimize images** before uploading as assets +6. **Minimize GraphQL response size** with specific field selection + +### Error Handling + +```liquid +{% graphql result = 'create_record', name: name %} + +{% if result.record_create.errors %} +
+ {% for error in result.record_create.errors %} +

{{ error.message }}

+ {% endfor %} +
+{% else %} +

Success! ID: {{ result.record_create.id }}

+{% endif %} +``` + +--- + +## 17. Common Gotchas & Pitfalls + +### 1. Variable Scope in Background Jobs + +**WRONG:** +```liquid +{% assign user_id = context.current_user.id %} +{% background %} + {{ user_id }} {# nil - not passed #} +{% endbackground %} +``` + +**CORRECT:** +```liquid +{% assign user_id = context.current_user.id %} +{% background user_id: user_id %} + {{ user_id }} {# Works! #} +{% endbackground %} +``` + +### 2. N+1 Query Problem + +**WRONG (N+1 queries):** +```liquid +{% graphql companies = 'get_companies' %} +{% for company in companies.records.results %} + {% graphql programmers = 'get_programmers', company_id: company.id %} + {# Each iteration = 1 query! #} +{% endfor %} +``` + +**CORRECT (single query):** +```graphql +query get_companies_with_programmers { + records( + filter: { table: { value: "company" } } + ) { + results { + id + properties + programmers: related_records( + table: "programmer" + foreign_property: "company_id" + ) { + id + properties + } + } + } +} +``` + +### 3. Form Field Name Format + +**WRONG:** +```liquid + {# Won't bind to form #} +``` + +**CORRECT:** +```liquid + +``` + +### 4. Module File References + +**WRONG:** +```liquid +{% render 'modules/my_module/public/header' %} +``` + +**CORRECT:** +```liquid +{% render 'modules/my_module/header' %} +``` + +### 5. Date/Time Formatting + +**WRONG:** +```liquid +{{ '2024-01-01' | strftime: '%Y' }} {# Error - not a time object #} +``` + +**CORRECT:** +```liquid +{{ '2024-01-01' | to_time | strftime: '%Y' }} +``` + +### 6. Array vs JSONB Confusion + +**Arrays** - for simple lists: +```yaml +type: array +# Value: ["a", "b", "c"] +``` + +**JSONB** - for complex objects: +```yaml +type: jsonb +# Value: {"nested": {"key": "value"}} +``` + +### 7. Form Resource Owner + +**For public forms** (contact, newsletter): +```yaml +resource_owner: anyone +``` + +**For authenticated forms** (profile edit): +```yaml +resource_owner: self +``` + +**For admin forms**: +```yaml +resource_owner: anyone_with_token +authorization_policies: + - admin_only_policy +``` + +### 8. Whitespace in Liquid + +**Problem:** Extra whitespace in output +```liquid +{% if true %} + Content +{% endif %} +{# Outputs newlines around content #} +``` + +**Solution:** Use whitespace control +```liquid +{%- if true -%} + Content +{%- endif -%} +``` + +### 9. GraphQL Variable Types + +**Integer vs Float:** +```graphql +# Integer property +{ name: "count", value_int: 5 } + +# Float property +{ name: "price", value_float: 19.99 } +``` + +**Boolean:** +```graphql +{ name: "active", value_boolean: true } +``` + +### 10. Soft Delete vs Hard Delete + +**Soft delete** (default): +```graphql +mutation { + record_delete(id: "123") { + id + deleted_at # Timestamp set + } +} +``` + +**Hard delete** (permanent): +```graphql +mutation { + record_delete(id: "123", hard_delete: true) { + id + } +} +``` + +### 11. Reserved Names + +Avoid these reserved names for custom tables and properties: + +**System Fields (automatically created):** +- `id` - Record UUID +- `created_at` - Creation timestamp +- `updated_at` - Last update timestamp +- `deleted_at` - Soft delete timestamp +- `type_name` - Table name +- `properties` - Property container + +**Reserved Words:** +- `user`, `users` - Built-in User table +- `session`, `sessions` - Session management +- `record`, `records` - Record operations +- `constant`, `constants` - System constants +- `table`, `tables` - Table metadata + +### 12. Form Resource Owner Confusion + +| Value | When to Use | +|-------|-------------| +| `anyone` | Public forms (contact, newsletter) | +| `self` | User editing their own data | +| `anyone_with_token` | API endpoints with token auth | + +**Wrong:** +```yaml +resource_owner: self # Won't work for public contact form +``` + +**Correct:** +```yaml +resource_owner: anyone # For public forms +``` + +### 13. Module File Deletion Behavior + +By default, module files are **NOT deleted** during deploy to protect private files. + +To enable deletion for a module: +```yaml +# app/config.yml +modules_that_allow_delete_on_deploy: + - my_module +``` + +### 14. GraphQL Query Caching + +GraphQL queries are cached by default. To bypass cache: +```graphql +query { + records( + per_page: 10 + filter: { table: { value: "product" } } + ) @skip_cache { + results { id } + } +} +``` + +### 15. File Upload Size Limits + +| Upload Type | Max Size | +|-------------|----------| +| Direct S3 (single part) | 5 GB | +| Direct S3 (multipart) | 5 TB | +| Application-processed | 2 GB | + +### 16. Background Job Payload Limits + +```liquid +{# WRONG - payload too large #} +{% background data: huge_array_with_thousands_of_items %} + +{# CORRECT - pass reference only #} +{% background record_id: record_id %} + {% graphql record = 'get_record', id: record_id %} + {# Process data in background #} +{% endbackground %} +``` + +### 17. Liquid Truthiness + +In Liquid, only `nil` and `false` are falsy. Empty strings and zero are truthy: + +```liquid +{% if '' %}TRUE{% endif %} {# TRUE! #} +{% if 0 %}TRUE{% endif %} {# TRUE! #} +{% if empty_array %}TRUE{% endif %} {# FALSE (nil) #} +{% if false %}TRUE{% endif %} {# FALSE #} +``` + +Use `blank` and `present` for better checks: +```liquid +{% if '' == blank %}EMPTY{% endif %} {# EMPTY #} +{% if 0 == blank %}ZERO IS BLANK{% endif %} {# Not blank! #} +``` + +--- + +## 18. Performance Optimization + +### Measuring Performance + +**time_diff filter:** +```liquid +{% assign start = 'now' | to_time %} + +{% graphql posts = 'get_posts' %} + +{% assign duration = start | time_diff: 'now' %} +

Query took: {{ duration }}ms

+``` + +### Query Optimization + +**1. Select only needed fields:** +```graphql +# BAD - fetches everything +query { + records { results { properties } } +} + +# GOOD - specific fields +query { + records { + results { + id + properties + } + } +} +``` + +**2. Use pagination:** +```graphql +query { + records(per_page: 20, page: 1) { + total_entries + results { id } + } +} +``` + +**3. Load related records efficiently:** +```graphql +query { + records(filter: { table: { value: "order" } }) { + results { + id + items: related_records(table: "order_item") { + id + properties + } + } + } +} +``` + +### Caching Strategies + +**Static Cache (Edge Caching):** +```liquid +--- +slug: public-page +response_headers: + Cache-Control: public, max-age=3600 +--- +``` + +**Fragment Caching:** +```liquid +{% cache key: 'sidebar', expire: 3600 %} + {% graphql categories = 'get_categories' %} + {% for category in categories.records.results %} + {{ category.properties.name }} + {% endfor %} +{% endcache %} +``` + +### Background Job Optimization + +**Keep payloads small:** +```liquid +{# BAD - large payload #} +{% background data: huge_array %} + +{# GOOD - pass reference #} +{% assign job_id = 'process_' | append: record_id %} +{% background job_id: job_id, record_id: record_id %} + {% graphql record = 'get_record', id: record_id %} + {# Process in background #} +{% endbackground %} +``` + +--- + +## 19. Testing & CI/CD + +### pos-cli GUI + +```bash +# Start GUI for GraphQL development +pos-cli gui serve staging + +# Access at http://localhost:3333 +``` + +### platformOS Check + +```bash +# Install +npm install -g @platformos/platformos-check + +# Run checks +platformos-check + +# Auto-fix issues +platformos-check --auto-correct +``` + +### GitHub Actions CI + +**File:** `.github/workflows/platformos.yml` +```yaml +name: platformOS CI + +on: + push: + branches: [main] + pull_request: + branches: [main] + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Setup Node.js + uses: actions/setup-node@v3 + with: + node-version: '18' + + - name: Install pos-cli + run: npm install -g @platformos/pos-cli + + - name: Deploy to Staging + run: pos-cli deploy staging + env: + MPKIT_TOKEN: ${{ secrets.MPKIT_TOKEN }} + MPKIT_URL: ${{ secrets.STAGING_URL }} + + - name: Run Tests + run: npm test +``` + +### Release Pool Setup + +1. Create dedicated test instances in Partner Portal +2. Configure GitHub secrets: + - `MPKIT_TOKEN` + - `STAGING_URL` + - `PRODUCTION_URL` + +### Testing Best Practices + +1. **Unit test** GraphQL queries +2. **Integration test** form submissions +3. **E2E test** critical user flows +4. **Performance test** with realistic data volumes +5. **Security test** authorization policies + +--- + +## 20. System Limitations + +### Resource Limits + +| Resource | Limit | Notes | +|----------|-------|-------| +| File upload size | 2GB | Assets and uploads | +| Background job payload | 100KB | Keep payloads small | +| Background job execution | 1-60 min | Depends on priority | +| GraphQL query complexity | Varies | Monitor performance | +| Records per query | Unlimited | Use pagination | +| Assets | Thousands | Use uploads for dynamic content | +| Uploads | Millions | No practical limit | + +### Background Job Limits + +| Priority | Max Execution | Use For | +|----------|---------------|---------| +| `high` | 1 minute | Critical, urgent tasks | +| `default` | 5 minutes | Standard operations | +| `low` | 60 minutes | Heavy processing | + +### Rate Limiting + +- API calls may be rate-limited based on plan +- Background job scheduling has queue limits +- GraphQL queries have complexity scoring + +### Reserved Names + +Avoid these names for custom tables/properties: +- `id`, `created_at`, `updated_at`, `deleted_at` +- `type_name`, `properties`, `user` +- Built-in Liquid objects and filters + +--- + +## 22. Data Import/Export + +### Exporting Data + +```bash +# Export all data from an instance +pos-cli data export staging --path=./export.json + +# Export specific tables +pos-cli data export staging --tables=products,orders --path=./products.json +``` + +### Importing Data + +```bash +# Import data to an instance +pos-cli data import staging ./export.json + +# Import with transformations +pos-cli data import staging ./data.json --transform=./transform.js +``` + +### Data Export Format + +```json +{ + "users": [ + { + "id": "123", + "email": "user@example.com", + "created_at": "2024-01-15T10:00:00Z", + "properties": { + "first_name": "John", + "last_name": "Doe" + } + } + ], + "records": { + "product": [ + { + "id": "456", + "properties": { + "name": "Widget", + "price": 19.99 + } + } + ] + } +} +``` + +### Programmatic Import with Migrations + +```liquid +{# app/migrations/20240115000000_import_products.liquid #} +{% parse_json data %} + {{ 'data/products.json' | load_file }} +{% endparse_json %} + +{% for product in data.products %} + {% graphql result = 'create_product', + name: product.name, + price: product.price, + sku: product.sku + %} + {% log result %} +{% endfor %} +``` + +### Cleaning Instance Data + +```bash +# WARNING: This deletes all data! +pos-cli data clean staging + +# Clean specific tables +pos-cli data clean staging --tables=products,orders +``` + +--- + +## 23. Quick Reference + +### File Templates + +**New Page:** +```liquid +--- +slug: my-page +layout: application +--- + +

Page Title

+``` + +**New Table:** +```yaml +name: my_table +properties: + - name: name + type: string +``` + +**New Form:** +```liquid +--- +name: my_form +resource: my_table +resource_owner: anyone +redirect_to: /success +fields: + properties: + name: + validation: + presence: true +--- + +{% form %} + + +{% endform %} +``` + +**New GraphQL Query:** +```graphql +query my_query($param: String) { + records(filter: { table: { value: "my_table" } }) { + results { id properties } + } +} +``` + +### Common Liquid Patterns + +**Conditional rendering:** +```liquid +{% if condition %} + +{% elsif other_condition %} + +{% else %} + +{% endif %} +``` + +**Loop with index:** +```liquid +{% for item in items %} + {{ forloop.index }}: {{ item.name }} +{% endfor %} +``` + +**Pagination:** +```liquid +{% if records.has_previous_page %} + Previous +{% endif %} + +{% if records.has_next_page %} + Next +{% endif %} +``` + +### Common GraphQL Patterns + +**Create with error handling:** +```graphql +mutation { + record_create(record: { table: "post", properties: [] }) { + id + errors { message } + } +} +``` + +**Update specific fields:** +```graphql +mutation { + record_update(id: "123", record: { properties: [{ name: "status", value: "published" }] }) { + id + properties + } +} +``` + +**Search with filters:** +```graphql +query { + records( + filter: { + table: { value: "product" } + properties: [{ name: "category", value: "electronics" }] + created_at: { gte: "2024-01-01" } + } + ) { + results { id } + } +} +``` + +### pos-cli Commands + +```bash +# Authentication +pos-cli auth login # Login to Partner Portal + +# Development +pos-cli sync staging # Watch and sync changes +pos-cli deploy staging # Deploy to instance +pos-cli deploy staging -f # Force deploy (delete missing files) + +# Data +pos-cli data export staging # Export instance data +pos-cli data import staging file.json # Import data +pos-cli migrations run staging # Run pending migrations + +# Modules +pos-cli modules install module_name # Install module +pos-cli modules remove module_name # Remove module + +# GUI +pos-cli gui serve staging # Start development GUI + +# Logs +pos-cli logs staging # Stream logs +``` + +### Error Messages Reference + +| Error | Cause | Solution | +|-------|-------|----------| +| `Record not found` | Invalid ID | Check record exists | +| `Validation failed` | Invalid data | Check form validations | +| `Unauthorized` | Policy failed | Check authorization | +| `Rate limited` | Too many requests | Add delays, use caching | +| `Timeout` | Query too slow | Optimize query, add pagination | +| `Property not found` | Wrong property name | Check table schema | +| `Table not found` | Wrong table name | Check table definition | +| `Form not found` | Wrong form name | Check form file exists | + +### GraphQL Property Type Mapping + +| Property Type | GraphQL Input | Example | +|---------------|---------------|---------| +| `string` | `value: "text"` | `{ name: "title", value: "Hello" }` | +| `integer` | `value_int: 42` | `{ name: "count", value_int: 5 }` | +| `float` | `value_float: 19.99` | `{ name: "price", value_float: 19.99 }` | +| `boolean` | `value_boolean: true` | `{ name: "active", value_boolean: true }` | +| `date` | `value: "2024-01-15"` | `{ name: "birthday", value: "2024-01-15" }` | +| `datetime` | `value: "2024-01-15T10:00:00Z"` | ISO 8601 format | +| `array` | `value_array: ["a", "b"]` | `{ name: "tags", value_array: ["a", "b"] }` | +| `jsonb` | `value_json: "{}"` | JSON string | +| `upload` | Via form only | File uploads | + +### Form Validation Reference + +| Validation | Syntax | Description | +|------------|--------|-------------| +| `presence` | `presence: true` | Required field | +| `email` | `email: true` | Valid email format | +| `uniqueness` | `uniqueness: true` | Must be unique | +| `length` | `length: { minimum: 5, maximum: 100 }` | String length | +| `numericality` | `numericality: { greater_than: 0 }` | Number range | +| `confirmation` | `confirmation: true` | Must match confirmation field | +| `url` | `url: true` | Valid URL format | + +### pos-cli Extended Commands + +```bash +# Authentication +pos-cli auth login # Login to Partner Portal +pos-cli auth logout # Logout + +# Development +pos-cli sync staging # Watch and sync changes +pos-cli sync staging --live-reload # With live reload +pos-cli deploy staging # Deploy to instance +pos-cli deploy staging -f # Force deploy (delete missing files) +pos-cli deploy staging --direct-assets # Deploy assets directly + +# Data Management +pos-cli data export staging # Export all data +pos-cli data export staging --tables=products,orders +pos-cli data import staging file.json # Import data +pos-cli data clean staging # Delete all data (DANGER!) +pos-cli migrations run staging # Run pending migrations +pos-cli migrations status staging # Check migration status + +# Modules +pos-cli modules install module_name # Install module +pos-cli modules install module_name@1.2 # Specific version +pos-cli modules remove module_name # Remove module +pos-cli modules list staging # List installed modules + +# GUI Tools +pos-cli gui serve staging # Start development GUI +pos-cli gui serve staging --port 3333 # Custom port + +# Logs +pos-cli logs staging # Stream logs +pos-cli logs staging --tail 100 # Last 100 lines +pos-cli logs staging --follow # Follow new logs + +# Environment +pos-cli env list # List environments +pos-cli env add production # Add environment +pos-cli env remove staging # Remove environment + +# Testing +pos-cli test staging # Run tests + +# Debug +pos-cli shell staging # Interactive shell +``` + +--- + +## 24. Translations + +### Overview + +Translations serve three main purposes: +1. **Multi-language sites** - Static copy in multiple languages +2. **Date formatting** - Consistent date/time display +3. **Flash messages** - System message localization + +### Translation Files + +**File:** `app/translations/en.yml` +```yaml +en: + hello: "Hello" + welcome: "Welcome to our site" + buttons: + submit: "Submit" + cancel: "Cancel" + errors: + not_found: "Page not found" +``` + +**File:** `app/translations/es.yml` +```yaml +es: + hello: "Hola" + welcome: "Bienvenido a nuestro sitio" + buttons: + submit: "Enviar" + cancel: "Cancelar" + errors: + not_found: "Página no encontrada" +``` + +### Using Translations in Liquid + +**Basic translation:** +```liquid +{{ 'hello' | t }} # Output: Hello (or Hola) +``` + +**Nested keys:** +```liquid +{{ 'buttons.submit' | t }} # Output: Submit +{{ 'errors.not_found' | t }} # Output: Page not found +``` + +**With interpolation:** +```yaml +# en.yml +welcome_user: "Welcome, {{ name }}!" +``` +```liquid +{{ 'welcome_user' | t: name: user.first_name }} +``` + +### Date Localization + +Use the `l` (localize) filter for consistent date formatting: + +```yaml +# en.yml +date: + formats: + short: "%b %d, %Y" + long: "%B %d, %Y %H:%M" +``` +```liquid +{{ 'now' | l: 'short' }} # Jan 15, 2024 +{{ post.published_at | l: 'long' }} # January 15, 2024 14:30 +``` + +### Language Detection + +platformOS automatically detects language from: +1. User's `language` property (if set) +2. Browser's Accept-Language header +3. Default language (English) + +Access current language: +```liquid +{{ context.language }} # Current language code (e.g., "en") +``` + +--- + +## 25. Activity Feeds + +### Overview + +Activity Feeds implement the [W3C Activity Streams 2.0](https://www.w3.org/TR/2017/REC-activitystreams-core-20170523/) specification for tracking user activities. + +**Key Characteristics:** +- Activities are **immutable** (append-only) +- Each activity has a **unique UUID** +- Activities can be shared between actors +- Activities represent events that happened in the past + +### Activity Structure + +```json +{ + "actor": { + "type": "Person", + "id": "User.1", + "name": "Sally Smith" + }, + "type": "Create", + "object": { + "type": "Relationship", + "id": "Relationship.42" + }, + "target": { + "type": "Group", + "id": "Group.5" + } +} +``` + +### Creating Activities + +**GraphQL Mutation:** +```graphql +mutation create_activity { + activity_create( + activity: { + type: "Join" + actor: { + type: "Person" + id: "User.123" + name: "John Doe" + } + object: { + type: "Group" + id: "Group.456" + } + } + ) { + id + uuid + } +} +``` + +### Publishing to Feeds + +After creating an activity, publish it to feeds: + +```graphql +mutation publish_to_feed { + feed_publish( + feed_id: "user_123_notifications" + activity_uuid: "abc-123-uuid" + ) { + id + } +} +``` + +### Querying Feeds + +```graphql +query get_user_feed { + feeds( + feed_id: "user_123_notifications" + per_page: 20 + ) { + total_entries + results { + id + uuid + type + actor + object + target + created_at + } + } +} +``` + +### Common Activity Types + +| Type | Description | +|------|-------------| +| `Create` | Created something | +| `Update` | Updated something | +| `Delete` | Deleted something | +| `Join` | Joined a group/event | +| `Leave` | Left a group/event | +| `Follow` | Started following | +| `Like` | Liked content | +| `Comment` | Commented on content | +| `Share` | Shared content | +| `Approve` | Approved a request | + +--- + +## 26. JSON Documents + +### Overview + +JSON Documents provide a schemaless data storage option for flexible, document-based data. Unlike Records (which require a Table schema), JSON Documents can store any valid JSON structure. + +**Use Cases:** +- Configuration data +- Unstructured content +- Temporary data storage +- Data that doesn't fit a rigid schema + +### Creating JSON Documents + +**GraphQL Mutation:** +```graphql +mutation create_json_document { + json_document_create( + document: { + name: "site_config" + content: "{\"theme\": \"dark\", \"features\": [\"blog\", \"shop\"]}" + } + ) { + id + name + content + created_at + } +} +``` + +### Querying JSON Documents + +```graphql +query get_json_document { + json_document(name: "site_config") { + id + name + content + created_at + updated_at + } +} + +query list_json_documents { + json_documents( + per_page: 10 + sort: [{ created_at: { order: DESC } }] + ) { + results { + id + name + content + } + } +} +``` + +### Updating JSON Documents + +```graphql +mutation update_json_document { + json_document_update( + name: "site_config" + document: { + content: "{\"theme\": \"light\", \"features\": [\"blog\", \"shop\", \"forum\"]}" + } + ) { + id + content + updated_at + } +} +``` + +### Using in Liquid + +```liquid +{% graphql config = 'get_json_document', name: 'site_config' %} +{% assign settings = config.json_document.content | parse_json %} + +Theme: {{ settings.theme }} +Features: {{ settings.features | join: ', ' }} +``` + +### JSON Document vs Records + +| Feature | JSON Documents | Records | +|---------|---------------|---------| +| Schema | Schemaless | Defined in Table YAML | +| Validation | None | Form validation | +| Structure | Any JSON | Fixed properties | +| Use Case | Config, flexible data | Structured entities | +| GraphQL | `json_document_*` | `record_*` | + +--- + +## 27. AI Embeddings + +### Overview + +platformOS supports AI embeddings for semantic search and similarity matching. Embeddings are vector representations of text that capture semantic meaning. + +**Use Cases:** +- Semantic search +- Content recommendation +- Similarity matching +- Clustering + +### Creating Embeddings + +**GraphQL Mutation:** +```graphql +mutation create_embedding { + embedding_create( + embedding: { + name: "product_description" + value: "High-quality wireless headphones with noise cancellation" + target_id: "product_123" + target_type: "Product" + } + ) { + id + vector + } +} +``` + +### Semantic Search + +```graphql +query semantic_search { + embeddings_search( + query: "wireless audio devices" + limit: 10 + threshold: 0.7 + ) { + results { + id + target_id + target_type + similarity + value + } + } +} +``` + +### Querying Embeddings + +```graphql +query get_embedding { + embedding( + target_id: "product_123" + target_type: "Product" + ) { + id + name + value + vector + created_at + } +} +``` + +### Deleting Embeddings + +```graphql +mutation delete_embedding { + embedding_delete( + target_id: "product_123" + target_type: "Product" + ) { + id + } +} +``` + +### Embedding Parameters + +| Parameter | Description | +|-----------|-------------| +| `name` | Identifier for the embedding type | +| `value` | The text to embed | +| `target_id` | ID of the associated entity | +| `target_type` | Type of the associated entity | +| `vector` | The computed embedding vector (read-only) | + +--- + +## 28. Migrations + +### Overview + +Migrations are Liquid scripts that run once to transform data. They are useful for: +- Data transformations during schema changes +- Bulk data updates +- One-time data imports + +### Creating Migrations + +**File:** `app/migrations/20240115120000_add_status_to_products.liquid` +```liquid +{% graphql products = 'get_all_products' %} + +{% for product in products.records.results %} + {% graphql result = 'update_product_status', + id: product.id, + status: 'active' + %} + {% log result %} +{% endfor %} +``` + +### Migration File Naming + +Migrations are executed in alphabetical order. Use timestamps as prefixes: +``` +app/migrations/ +├── 20240101000000_initial_setup.liquid +├── 20240115120000_add_status.liquid +└── 20240201000000_migrate_images.liquid +``` + +### Running Migrations + +```bash +# Run pending migrations +pos-cli migrations run staging + +# Check migration status +pos-cli migrations status staging +``` + +### Migration Best Practices + +1. **Make migrations idempotent** - Running twice should not cause errors: +```liquid +{% graphql product = 'get_product', id: product_id %} +{% unless product.record.properties.status %} + {# Only update if status is not set #} + {% graphql result = 'update_product', id: product_id, status: 'active' %} +{% endunless %} +``` + +2. **Use background jobs for large migrations:** +```liquid +{% background source_name: 'data_migration' %} + {% graphql records = 'get_all_records' %} + {% for record in records.records.results %} + {# Process each record #} + {% endfor %} +{% endbackground %} +``` + +3. **Test migrations on staging first** +4. **Log progress for debugging:** +```liquid +{% log 'Migration started' %} +{% log 'Processed ' | append: count | append: ' records' %} +``` + +### Migration Limitations + +- Migrations run as background jobs +- Should complete within a few minutes +- For long-running operations, use low-priority background jobs +- Failed migrations can be retried + +--- + +## Resources + +- **Documentation:** https://documentation.platformos.com/ +- **API Reference:** https://documentation.platformos.com/api-reference +- **Examples:** https://examples.platform-os.com/ +- **GitHub:** https://github.com/Platform-OS +- **Partner Portal:** https://partners.platformos.com/ +- **Community:** https://community.platformos.com/ + +--- + +*This guide is designed for LLM agents developing on platformOS. For the most up-to-date information, always refer to the official documentation.* diff --git a/src/data/resources/short-platformos-development-guide.md b/src/data/resources/short-platformos-development-guide.md new file mode 100644 index 0000000..d2a17dd --- /dev/null +++ b/src/data/resources/short-platformos-development-guide.md @@ -0,0 +1,1079 @@ +# platformOS Development Guide + +Every rule uses MUST/MUST NOT. No information omitted. Section 0 is the mandatory +workflow — read it before touching any file. + +## 0. MANDATORY WORKFLOW — Read Before Writing Any Code + +**You are STRICTLY FORBIDDEN from skipping this workflow** + +You MUST follow this loop for every feature. Each step produces structured output +the next step consumes — skipping any step produces invalid state that downstream +tools will reject. + +1. **`project_map`** — understand what already exists. MUST be called once per session + before any scaffold or write. +2. **`scaffold(type, name, properties, write: false)`** — generate the authoritative + file set from platformOS-native templates. MUST use scaffold whenever a file set + matches one of its types (crud, api, command, query, partial, page). +3. **`domain_guide(domain)` for every domain in your plan** — BEFORE drafting files. + Skipping this is the #1 cause of broken platformOS code. `domain_guide` contains + rules that are NOT in your training data and that differ from Shopify, Rails, and + generic Liquid. +4. **`validate_intent` — declare your plan before touching disk.** + Two modes, pick by what you're doing next: + + - **Mode A — hand-drafted batch (REQUIRED before manual writes).** + Call `validate_intent({ intent: { goal, changes: [...] } })` where + `changes` is an array of `{ path, role, action, references? }` — one + entry per file you intend to author. The plan is the contract for the + rest of the session. + - **Mode B — scaffold review (OPTIONAL).** + Call `validate_intent({ scaffold_output: })` + only if you want a second look at the generated set before committing. + The default scaffold path skips this step. + + **Read the response:** + - `ok: false` → fix `errors[].suggestion`, re-call. MUST NOT proceed. + - `ok: true` + `write_directly: true` → Mode B; go straight to + `scaffold(..., write: true)`. + - `ok: true` + `write_directly: false` → Mode A; draft each file, call + `validate_code` on the full content, then write. + + **What `pending_files` / `pending_translations` / `pending_pages` are for:** + you can ignore them. The supervisor stores them and uses them to suppress + false-positive `MissingPartial` / `TranslationKeyExists` errors in later + `validate_code` / `analyze_project` calls — because those files are + *promised* by the plan but not on disk yet. You do not pass them to any + subsequent tool; the server merges them automatically. + + **Skipping Mode A before hand-drafted writes** is the #1 cause of phantom + cross-reference errors: `validate_code` will flag every partial and + translation key the plan hasn't written yet, and the agent chases those + ghosts by deleting the references the plan needs. + + **Scope drift:** if you add, rename, or drop a file that isn't in the + current `changes` array, re-call `validate_intent` with the updated plan + before writing the new file. + +5. **`scaffold(..., write: true)`** — writes all files to disk. If you went + through Mode B in step 4, this runs after `write_directly: true`. + Otherwise this is the direct follow-up to step 2. For hand-drafted edits + (Mode A, or manual edits without scaffold), call `validate_code` per file + and only write when validation passes — never rely on scaffold to write a + hand-authored file. +6. **Feedback loop.** When `validate_code` returns `status !== "ok"` or + `must_fix_before_write: true`, fix every error and re-validate. MUST NOT + write the file to disk until validation passes. + When debugging existing files, always read them from disk first and submit + their actual content to `validat_code` tool. +7. Creation order matters: schema → graphql → partial → page. +8. **`analyze_project` — project-wide health check.** MUST be called: + - **Before reporting task completion.** `validate_code` only sees one + file at a time; cross-file damage (broken render targets, orphaned + partials, dangling translations, schema drift) only surfaces from the + whole-project view. A task is not done until `analyze_project` returns + zero new errors or warnings introduced by this session. + - **When you feel lost.** If validate_code keeps reporting errors you + don't understand, if the same check keeps re-appearing after you + "fixed" it, if you suspect a file you edited affected callers you + can't see, or if `project_map` no longer matches your mental model — + stop editing and call `analyze_project` to re-ground. It returns + per-file error counts, the dependency graph, orphaned files, broken + references, and schema issues for every file in `app/`. That is the + authoritative picture of the project right now. + + `analyze_project` respects `session.pending` — files declared in a + validated plan are not flagged as missing. You do not need to pass any + parameters for the standard case; omit `files` to analyze the whole + project. + + MUST NOT: skip this step before announcing "done" just because + `validate_code` passed on the files you edited. Individual-file green + lights do not imply project integrity. + + +### MUST-CALL domains (by feature type) + +- **Auth code** — `domain_guide(domain: "authentication")` +- **Any form** — `domain_guide(domain: "forms")` +- **New pages** — `domain_guide(domain: "pages")` +- **New partials** — `domain_guide(domain: "partials")` +- **GraphQL ops** — `domain_guide(domain: "graphql")` +- **Any new domain** — `domain_guide(domain: "", section: "gotchas")` + +### MUST NOT + +- Use `{% include %}` for app code — deprecated. Use `{% render %}` or + `{% function %}`. +- Use Shopify objects (`shop`, `cart`, `customer`, `product`, `collection`). These + do not exist in platformOS. +- Write hand-drafted files to disk without calling `validate_code` on the proposed + content first. (Scaffold-written files are exempt — they are pre-validated.) +- Assume module call syntax from memory — call `module_info(name)` to get the + authoritative live-scan API surface. +- Ignore `consult_before_writing` in a scaffold response. Every domain listed there + MUST be consulted via `domain_guide` before writing. + +### Session-start checklist + +Before your first tool call, the following are true: + +- [ ] `server_status` called — confirms LSP and indexes are ready, lists + `domain_guides` and `session_pending`. +- [ ] `load_development_guide` called (this document) — re-read if you lose + context or are unsure which step comes next. +- [ ] `project_map` called once for full project baseline. + +Proceed only when all three are checked. + +## 1. Technology Stack + +platformOS uses three primary technologies: +- **Liquid** — server-side templating language +- **GraphQL** — data operations (built-in queries/mutations only) +- **YAML** — configuration for schemas, translations, and settings + +The underlying databases (PostgreSQL, ElasticSearch, Redis) MUST be accessed ONLY through GraphQL and Liquid. There is NO direct database access. + +platformOS does NOT provide public GraphQL endpoints for client-side access. All GraphQL operations MUST be executed server-side using the `{% graphql %}` Liquid tag. + +### Source of Truth + +The official platformOS documentation is the ONLY source of truth: + +| Resource | URL | +|----------|-----| +| Official Docs | documentation.platformos.com | +| GraphQL Schema | documentation.platformos.com/api/graphql/schema | +| Liquid Filters | documentation.platformos.com/api-reference/liquid/platformos-filters.md | +| Liquid Tags | documentation.platformos.com/api-reference/liquid/platformos-tags.md | +| Context Object | documentation.platformos.com/api-reference/liquid/platformos-objects.md | +| Core Module | github.com/Platform-OS/pos-module-core (README) | +| User Module | github.com/Platform-OS/pos-module-user (README) | +| Common Styling | github.com/Platform-OS/pos-module-common-styling (README) | +| Payments Module | github.com/Platform-OS/pos-module-payments (README) | +| Payments Stripe | github.com/Platform-OS/pos-module-payments-stripe (README) | +| Tests Module | github.com/Platform-OS/pos-module-tests (README) | +| Migrations | documentation.platformos.com/developer-guide/data-import-export/migrating-data.md | + +You MUST NOT invent undocumented behaviors, APIs, configurations, or directory structures. When uncertain, consult documentation. + +--- + +## 2. Directory Structure + +``` +project-root/ +├── app/ +│ ├── assets/ # Static files (images, fonts, styles, scripts) +│ ├── views/ +│ │ ├── pages/ # Controllers — NO HTML here +│ │ ├── layouts/ # Wrapper templates +│ │ └── partials/ # Reusable template snippets +│ ├── lib/ +│ │ ├── commands/ # Business logic (build → check → execute) +│ │ ├── queries/ # Data retrieval wrappers +│ │ ├── events/ # Event definitions +│ │ └── consumers/ # Event handlers +│ ├── schema/ # Database table definitions (YAML) +│ ├── graphql/ # GraphQL query/mutation files +│ ├── emails/ # Email templates +│ ├── smses/ # SMS templates +│ ├── api_calls/ # Third-party API integrations +│ ├── translations/ # i18n content (YAML) +│ ├── authorization_policies/ # DO NOT USE — use pos-module-user +│ ├── migrations/ # One-time migration scripts +│ └── config.yml # Feature flags +├── modules/ # Downloaded/custom modules (READ-ONLY) +└── .pos # Environment endpoints +``` + +All application files MUST reside in the `app/` directory. You MUST NOT create or modify application files outside `app/`. + +The `modules/` directory is READ-ONLY. You MUST NOT edit files in `modules/` — override via documented mechanisms only. + +### File Naming Conventions + +| Directory | Pattern | Example | +|-----------|---------|---------| +| Commands | `app/lib/commands//.liquid` | `app/lib/commands/questions/create.liquid` | +| Queries | `app/lib/queries//.liquid` | `app/lib/queries/articles/find.liquid` | +| Unit Tests | `app/lib/tests//_test.liquid` | `app/lib/tests/articles/create_test.liquid` | +| Pages | `app/views/pages//.liquid` | `app/views/pages/posts/show.liquid` | +| Partials | `app/views/partials//.liquid` | `app/views/partials/articles/card.liquid` | +| Assets | `app/assets//` | `app/assets/images/logo.png` | +| Translations | `app/translations/.yml` | `app/translations/en.yml` | + +### File Formats + +| Extension | Content-Type | URL | +|-----------|--------------|-----| +| `*.liquid` or `*.html.liquid` | `text/html` | `/path` | +| `*.json.liquid` | `application/json` | `/path.json` | +| `*.js.liquid` | `application/javascript` | `/path.js` | + +--- + +## 3. Architecture Rules + +### Pages MUST Be Controllers + +Pages MUST contain NO HTML, JS, or CSS. Pages MUST ONLY fetch data and delegate to partials via `render`. Each page file MUST handle exactly ONE HTTP method. + +### Business Logic MUST Live in Commands + +All business logic MUST reside in `app/lib/commands/`. Pages MUST delegate to commands. Commands MUST follow the build → check → execute pattern. + +### Path Resolution + +- `{% render 'blog_posts/card' %}` → `app/views/partials/blog_posts/card.liquid` +- `{% function r = 'commands/blog_posts/create' %}` → `app/lib/commands/blog_posts/create.liquid` +- `{% function r = 'queries/blog_posts/search' %}` → `app/lib/queries/blog_posts/search.liquid` + +The `lib/` prefix is implicit in `function` calls — do NOT include it. + +### Separation of Concerns + +- UI (Liquid templates) MUST be in partials and layouts +- Data operations (GraphQL) MUST be in query/mutation files +- Logic (commands) MUST be in `app/lib/commands/` + +### Modules First + +Every new feature MUST be built on top of existing platformOS modules (Core, User, Common-Styling, Test). You MUST NOT create duplicate models or authentication logic. + +### Generators First (DEPRECATED — DO NOT USE) + +You MUST prefer `pos-cli` generators (`generators-list`, `generators-run`) over manual file creation when available. + +--- + +## 4. Pages + +Pages are controllers — they handle routing, fetch data, and delegate to partials. + +### Front Matter + +```liquid +--- +slug: products/:id +method: post +layout: application +metadata: + title: "Product Details" +--- +``` + +| Property | Default | Notes | +|----------|---------|-------| +| `slug` | From file path | Supports `:param`, `*wildcard`, `(/:optional)` | +| `method` | `get` | `get`, `post`, `put`, `delete` | +| `layout` | `application` | Empty string for no layout | + +**You MUST NOT use `authorization_policies` in front matter — use User Module helpers instead.** +**For the home page (root /), omit the slug entirely — app/views/pages/index.liquid serves / by default.** +**For the home page omit method as it can only be `get` which is default.** +**One REST method per page** + +### Dynamic Routes + +| Pattern | URL | `context.params` | +|---------|-----|------------------| +| `products/:id` | `/products/123` | `{ "id": "123" }` | +| `files/*path` | `/files/a/b.txt` | `{ "path": "a/b.txt" }` | +| `search(/:q)` | `/search/books` | `{ "q": "books" }` | + +### REST CRUD Convention + +| HTTP Method | URL Slug | Page File | GraphQL | Purpose | +|-------------|----------|-----------|---------|---------| +| GET | `/posts/new` | `pages/posts/new.liquid` | — | Render create form | +| POST | `/posts` | `pages/posts/create.liquid` | `record_create` | Persist new resource | +| GET | `/posts/:id` | `pages/posts/show.liquid` | find query | Show single resource | +| GET | `/posts/:id/edit` | `pages/posts/edit.liquid` | find query | Render edit form | +| PUT/PATCH | `/posts/:id` | `pages/posts/update.liquid` | `record_update` | Update resource | +| DELETE | `/posts/:id` | `pages/posts/delete.liquid` | `record_delete` | Delete resource | +| GET | `/posts` | `pages/posts/index.liquid` | search query | List resources | + +### CSRF Protection + +Non-GET requests require a CSRF token. Without it, the platform cannot authenticate the request (user module queries return anonymous). + +### GET Page Example + +```liquid +--- +slug: articles/:id +method: get +--- +{% liquid + function article = 'queries/articles/find', id: context.params.id + + if article == blank + render '404' + break + endif + + render 'articles/show', article: article +%} +``` + +### POST Page Example + +```liquid +--- +slug: articles +method: post +--- +{% liquid + function result = 'commands/articles/create', object: context.params.article + + if result.valid + function _ = 'modules/core/commands/session/set', key: 'sflash', value: 'app.articles.created', from: context.location.pathname + redirect_to '/articles' + else + render 'articles/new', result: result + endif +%} +``` + +--- + +## 5. Partials & Layouts + +### Partials + +Partials MUST NOT contain hardcoded user-facing text — always use translations (`{{ 'app.key' | t }}`). + +Partials MUST NOT have underscore-prefixed filenames. + +The render path maps: `render 'path/name'` → `app/views/partials/path/name.liquid`. + +### Layouts + +The default layout is `application`. Set `layout: ""` (empty string) in front matter for no layout. + +--- + +## 6. Commands (Business Logic) + +All business logic MUST be encapsulated in commands following the build → check → execute pattern. + +### Main Command + +```liquid +{% doc %} + @param object {object} - Article data +{% enddoc %} + +{% liquid + function object = 'commands/articles/create/build', object: object + function object = 'commands/articles/create/check', object: object + + if object.valid + function object = 'modules/core/commands/execute', mutation_name: 'articles/create', selection: 'record', object: object + endif + + return object +%} +``` + +### Build Stage + +Normalizes and structures input data: + +```liquid +{% doc %} + @param object {object} - form params +{% enddoc %} + +{% liquid + assign object['title'] = object.title + assign object['body'] = object.body + + return object +%} +``` + +### Check Stage + +Validates the built object: + +```liquid +{% doc %} + @param object {object} - form params +{% enddoc %} + +{% liquid + assign c = '{ "errors": {}, "valid": true }' | parse_json + + function c = 'modules/core/lib/validations/presence', c: c, field_name: 'title', object: object + function c = 'modules/core/lib/validations/presence', c: c, field_name: 'body', object: object + + assign object = object | hash_merge: valid: c.valid, errors: c.errors + + return object +%} +``` + +### ~~Alternative Core Module Syntax~~ (DEPRECATED — DO NOT USE) + +> **Warning:** `modules/core/commands/build` and `modules/core/commands/check` do NOT exist in the core module. Only `modules/core/commands/execute` is a shared core command. Build and check MUST be per-model files (e.g., `commands/articles/create/build.liquid`, `commands/articles/create/check.liquid`). + +```liquid +{% comment %} WRONG — these partials do not exist: {% endcomment %} +{% function object = 'modules/core/commands/build', object: object %} +{% function object = 'modules/core/commands/check', object: object, + validators: '[{"name": "presence", "property": "title"}]' +%} + +{% comment %} CORRECT — only execute is shared: {% endcomment %} +{% if object.valid %} + {% function object = 'modules/core/commands/execute', + mutation_name: 'products/create', selection: 'record', object: object + %} +{% endif %} + +{% return object %} +``` + +### Events + +```liquid +{% comment %} Publish an event {% endcomment %} +{% function _ = 'modules/core/commands/events/publish', type: 'order_created', object: order %} + +{% comment %} Consumer: app/lib/consumers/order_created/send_email.liquid {% endcomment %} +{% graphql _ = 'emails/send_confirmation', email: event.object.email %} +``` + +All inputs MUST be validated in commands before persisting. + +--- + +## 7. GraphQL + +GraphQL MUST be called from pages, query wrappers (`app/lib/queries/`), or commands (via `modules/core/commands/execute`). You MUST NOT call GraphQL from partials/views. Raw GraphQL MUST NOT appear in pages — use `.graphql` files exclusively. + +### Query Wrapper Pattern + +```liquid +{% doc %} + @param id {string} - Article ID +{% enddoc %} + +{% liquid + graphql result = 'articles/find', id: id + return result.records.results | first +%} +``` + +### Search with Pagination + +```graphql +query search($page: Int = 1, $keyword: String) { + records( + page: $page + per_page: 20 + filter: { + table: { value: "article" } + properties: [{ name: "title", contains: $keyword }] + } + sort: { created_at: { order: DESC } } + ) { + total_pages + results { + id + title: property(name: "title") + body: property(name: "body") + } + } +} +``` + +All list queries MUST support `per_page` and `page` arguments for pagination. + +### Find by ID + +```graphql +query find($id: ID!) { + records( + per_page: 1 + filter: { + id: { value: $id } + table: { value: "article" } + } + ) { + results { + id + title: property(name: "title") + } + } +} +``` + +### Related Records (Avoids N+1) + +```graphql +results { + id + # belongs-to (single) + author: related_record(table: "user", join_on_property: "user_id") { + email + } + # has-many + comments: related_records(table: "comment", join_on_property: "id", foreign_property: "article_id") { + body: property(name: "body") + } +} +``` + +### Upload Property + +```graphql +image: property_upload(name: "image") { url } +``` + +### Mutations + +All mutations MUST alias the result as `record:` so `modules/core/commands/execute` can extract it with `selection: 'record'`: + +- `record: record_create(record: { table: "...", properties: [...] }) { id }` +- `record: record_update(id: $id, record: { properties: [...] }) { id }` +- `record: record_delete(table: "...", id: $id) { id }` — **`table` is required**, without it: runtime error "You must specify table" + +### Pagination Component + +```liquid +{% graphql result = 'products/search', page: context.params.page %} +{% render 'modules/common-styling/pagination', total_pages: result.records.total_pages %} +``` + +--- + +## 8. Schema + +Schema files define database tables in YAML at `app/schema/`. + +```yaml +# app/schema/article.yml +name: article +properties: + - name: title + type: string + - name: body + type: text + - name: published_at + type: datetime + - name: image + type: upload + options: + acl: public +``` + +### Property Types + +`string`, `text`, `integer`, `float`, `boolean`, `datetime`, `date`, `array`, `upload` + +--- + +## 9. Liquid Reference + +### Tags + +```liquid +{% graphql result = 'query_name', arg: value %} +{% function result = 'path/to/partial', arg: value %} +{% render 'partial', var: value %} +{% doc %} @param name {Type} - description {% enddoc %} +{% return result %} +{% export my_var, namespace: 'my_ns' %} +{% parse_json data %}{"key": "value"}{% endparse_json %} +{% redirect_to '/path', status: 302 %} +{% session key = value %} +{% log variable, type: 'debug' %} +{% cache 'key', expire: 3600 %}...{% endcache %} +{% background source_name: 'job_name', priority: 'low' %}...{% endbackground %} +{% content_for_layout %} +{% theme_render_rc 'modules/common-styling/toasts' %} +``` + +**`include` is DEPRECATED** — use `render` (UI partials) or `function` (logic partials) instead. Some module APIs still use `include` as their calling convention (follow those docs as-is). + +### Output + +```liquid +{{ variable }} +{{ variable | html_safe }} +{% print variable %} +``` + +### Common Filters + +- **Arrays:** `array_add`, `array_map`, `array_sort_by`, `array_group_by` +- **Hashes:** `hash_merge`, `hash_dig`, `hash_keys` +- **Dates:** `add_to_time`, `localize`, `is_date_in_past` +- **Validation:** `is_email_valid`, `is_json_valid` +- **Encoding:** `json`, `base64_encode`, `url_encode` + +### Coding Standards + +You MUST NOT line-wrap statements within `{% liquid %}` blocks. Each statement MUST be on a single line. + +**Correct:** +```liquid +{% liquid + assign filtered = products | where: 'available', true | map: 'title' | first + assign price = product | where: 'id', pid | map: 'price' | first +%} +``` + +**WRONG (causes syntax errors):** +```liquid +{% liquid + assign filtered = products + | where: 'available', true + | map: 'title' + | first +%} +``` + +--- + +## 10. Global Context + +**All global objects MUST use the `context.` prefix.** Using bare names (e.g., `params` instead of `context.params`, `page` instead of `context.page`) will fail silently or produce wrong results. + +| Property | Description | +|----------|-------------| +| `context.params` | HTTP parameters (query string + body) | +| `context.session` | Server-side session storage | +| `context.location` | URL info (`pathname`, `search`, `host`) | +| `context.environment` | `staging` or `production` | +| `context.is_xhr` | `true` for AJAX requests | +| `context.authenticity_token` | CSRF token | +| `context.constants` | Environment constants (hidden from `{{ context }}` for security) | +| `context.page.metadata` | Page metadata from front matter | + +You MUST NOT use `context.current_user` directly — always use `modules/user/queries/user/current`. + +--- + +## 11. User Module (Authentication & Authorization) + +You MUST use the User Module for all authentication and authorization. You MUST NOT use `authorization_policies/` directly. You MUST NOT duplicate login logic. You MUST NOT customize auth routes unless explicitly requested. + +### Built-in Roles + +- **Anonymous** — unauthenticated users +- **Authenticated** — any logged-in user +- **Superadmin** — bypasses ALL permission checks + +### Authorization Helpers + +```liquid +{% function profile = 'modules/user/queries/user/current' %} + +{% comment %} Check permission (returns true/false) {% endcomment %} +{% function can = 'modules/user/helpers/can_do', requester: profile, do: 'article.create' %} + +{% comment %} Enforce permission (403 if denied) — uses include (module API convention) {% endcomment %} +{% include 'modules/user/helpers/can_do_or_unauthorized', requester: profile, do: 'admin.view', redirect_anonymous_to_login: true %} + +{% comment %} Redirect if denied — uses include (module API convention) {% endcomment %} +{% include 'modules/user/helpers/can_do_or_redirect', requester: profile, do: 'orders.view', return_url: '/login' %} +``` + +> Note: These auth helpers use `include` because they need access to the caller's scope to halt execution. This is the module's documented API — do not replace with `render` or `function`. + +### Custom Permissions + +Override `modules/user/public/lib/queries/role_permissions/permissions.liquid`: + +```bash +mkdir -p app/modules/user/public/lib/queries/role_permissions +cp modules/user/public/lib/queries/role_permissions/permissions.liquid \ + app/modules/user/public/lib/queries/role_permissions/permissions.liquid +``` + +Define roles: +```liquid +{% parse_json data %} +{ + "admin": ["admin.view", "users.manage"], + "editor": ["article.create", "article.update"], + "superadmin": [] +} +{% endparse_json %} +{% return data %} +``` + +--- + +## 12. Core Module + +You MUST use pos-module-core for commands, events, and validators. + +--- + +## 13. Common Styling + +You MUST NOT use Tailwind, Bootstrap, or custom CSS frameworks. You MUST use `pos-*` prefixed classes from the common-styling module. Check `/style-guide` on your instance for available components. + +### Setup + +```liquid +{% comment %} In {% endcomment %} +{% render 'modules/common-styling/init' %} +``` +```html + +``` + +### File Upload Component + +```liquid +{% render 'modules/common-styling/forms/upload', + id: 'image', presigned_upload: presigned, name: 'image', + allowed_file_types: ['image/*'], max_number_of_files: 5 +%} +``` + +--- + +## 14. Translations (i18n) + +You MUST NOT hardcode user-facing text in partials. You MUST always use `{{ 'app.key' | t }}` and define translations in `app/translations/`. + +--- + +## 15. Forms + +You MUST use HTML `` tags. You MUST NOT use `{% form %}`. + +Forms MUST include the CSRF token: +```html + +``` + +For PUT/DELETE, forms MUST use POST with a `_method` hidden field: +```html + + + + + +``` + +Form fields MUST use bracket notation for resource binding: +```html + +``` + +Access in page: `context.params.resource` + +HTML forms submit checkbox values as \"on\" (string), but GraphQL expects boolean field to be Boolean type, not string. + +--- + +## 16. Constants & Credentials + +You MUST NOT hardcode API keys, secrets, or environment-specific URLs. You MUST use `context.constants`. + +### Setting Constants + +**Via CLI:** +```bash +pos-cli constants set --name STRIPE_SK_KEY --value "sk_test_..." dev +pos-cli constants set --name OPENAI_API_KEY --value "sk-..." dev +pos-cli constants set --name API_BASE_URL --value "https://api.example.com" dev +``` + +**Via GraphQL:** +```graphql +mutation { + constant_set(name: "STRIPE_SK_KEY", value: "sk_test_...") { + name + } +} +``` + +### Accessing Constants in Liquid + +Constants are hidden from `{{ context }}` for security. You MUST access them explicitly: +```liquid +{{ context.constants.STRIPE_SK_KEY }} +{{ context.constants.API_BASE_URL }} +``` + +### Naming Conventions + +| Use Case | Example | +|----------|---------| +| API keys | `STRIPE_SK_KEY`, `OPENAI_API_KEY`, `TWILIO_API_SECRET` | +| API URLs | `API_BASE_URL` | +| Feature flags | `FEATURE_NEW_CHECKOUT_ENABLED` | + +Staging constants SHOULD be initialized in migrations so new developers and tests can use test credentials automatically. + +--- + +## 17. Flash Messages & Toasts + +### Layout Setup (before ``) + +```liquid +{% liquid + function flash = 'modules/core/commands/session/get', key: 'sflash' + if context.location.pathname != flash.from or flash.force_clear + function _ = 'modules/core/commands/session/clear', key: 'sflash' + endif + render 'modules/common-styling/toasts', params: flash +%} +``` + +### Liquid Usage + +```liquid +{% liquid + function _ = 'modules/core/commands/session/set', key: 'sflash', value: 'app.order.confirmed', from: context.location.pathname + redirect_to '/orders' +%} +``` + +### JavaScript Usage + +```javascript +new pos.modules.toast('success', 'Saved!'); +new pos.modules.toast('error', 'Failed'); +``` + +--- + +## 18. Notifications (Email/SMS) + +```liquid +{% comment %} app/emails/order_confirmation.liquid {% endcomment %} +--- +to: {{ data.email }} +from: shop@example.com +subject: "Order #{{ data.order_id }}" +layout: mailer +--- +

Thank you for your order!

+``` + +Emails SHOULD be sent asynchronously using events + consumers. + +--- + +## 19. Payments (Stripe) + +### Install + +```bash +pos-cli modules install payments && pos-cli modules install payments_stripe +pos-cli constants set --name stripe_sk_key --value "sk_test_..." dev +``` + +### Create Transaction + +```liquid +{% function transaction = 'modules/payments/commands/transactions/create', + gateway: 'stripe', email: email, line_items: items, + success_url: '/thank-you', cancel_url: '/cart' +%} +{% function url = 'modules/payments/queries/pay_url', transaction: transaction %} +{% redirect_to url, status: 303 %} +``` + +Handle events via consumers: `payments_transaction_succeeded`, `payments_transaction_failed` + +**Test card:** `4242 4242 4242 4242`, any future date, any CVC. + +--- + +## 20. Migrations + +Migrations execute code outside the regular application cycle — useful for seeding data, initializing constants, and database modifications. + +### File Structure + +``` +app/migrations/ +├── 20240115120000_seed_initial_data.liquid +├── 20240116093000_add_default_categories.liquid +└── 20240120150000_init_staging_constants.liquid +``` + +Files MUST be named with UTC timestamp prefix for chronological execution. + +### Creating a Migration + +```bash +pos-cli migrations generate dev init_staging_constants +# Creates: app/migrations/YYYYMMDDHHMMSS_init_staging_constants.liquid +``` + +### Example: Initialize Staging Constants + +```liquid +{% liquid + if context.environment == 'staging' + graphql _ = 'constants/set', name: 'STRIPE_SK_KEY', value: 'sk_test_example123' + graphql _ = 'constants/set', name: 'API_BASE_URL', value: 'https://api-staging.example.com' + endif +%} +``` + +### Example: Seed Data + +```liquid +{% parse_json categories %} +["Electronics", "Clothing", "Books"] +{% endparse_json %} + +{% for category in categories %} + {% graphql _ = 'categories/create', name: category %} +{% endfor %} +``` + +### Running Migrations + +- **Automatic:** Pending migrations run on `pos-cli deploy` +- **Manual:** `pos-cli migrations run TIMESTAMP dev` + +### Migration States + +- **pending** — not yet executed (runs on next deploy) +- **done** — successfully completed (will not run again) +- **error** — failed (can edit and retry) + +For large data imports, use Data Import/Export instead of migrations. + +--- + +## 21. Testing + +Tests MUST go in `app/lib/tests/*_test.liquid`. Testing ONLY works in staging/development. + +Every new feature MUST have unit tests for commands. + +```liquid +{% function result = 'commands/products/create', title: "Test" %} +{% function contract = 'modules/tests/assertions/valid_object', contract: contract, object: result %} +{% function contract = 'modules/tests/assertions/equal', contract: contract, given: result.title, expected: "Test" %} +{% return contract %} +``` + +Run tests: `/_tests/run` in browser, or `pos-cli test run staging` for CI. + +--- + +## 22. CLI Commands + +```bash +# Deployment +pos-cli deploy dev + +# Sync (MUST sync every file after modification) +pos-cli sync dev + +# Logs +pos-cli logs dev + +# Linting (MUST run after EVERY file change) +platformos-check + +# Run Liquid inline +pos-cli exec liquid dev '' + +# Run GraphQL inline +pos-cli exec graphql dev '' + +# Tests +pos-cli test run staging + +# Modules +pos-cli modules install +pos-cli modules download + +# Constants +pos-cli constants set --name KEY --value "value" dev + +# Generate CRUD +pos-cli generate run modules/core/generators/crud --include-views + +# Migrations +pos-cli migrations generate dev +pos-cli migrations run TIMESTAMP dev +``` + +--- + +## 23. Modules Reference + +| Module | Install | Purpose | Required | +|--------|---------|---------|----------| +| `core` | Required | Commands, events, validators | YES | +| `user` | Required | Auth, RBAC, OAuth2 | YES | +| `common-styling` | Required | CSS, components | YES | +| `tests` | Optional | Testing framework | YES (for testing) | +| `payments` + `payments_stripe` | Optional | Stripe payments | No | +| `chat` | Optional | WebSocket messaging | No | +| `openai` | Optional | OpenAI integration | No | + +--- + +## 24. Forbidden Behaviors + +You MUST NOT: +- Edit files in `./modules/` (read-only) +- Break long lines in `{% liquid %}` blocks (causes syntax errors) +- Invent Liquid tags, filters, or GraphQL types that do not exist +- Use `{% form %}` tag (use HTML `
` only) +- Bypass security (CSRF tokens, authorization) +- Access databases directly outside GraphQL +- Deploy without running `platformos-check` +- Sync files outside `./app/` +- Use `authorization_policies/` directly (use pos-module-user) +- Use `context.current_user` directly (use user module queries) +- Use Tailwind, Bootstrap, or custom CSS frameworks (use common-styling) +- Hardcode API keys, secrets, or environment-specific URLs +- Hardcode user-facing text in partials (use translations) +- Put HTML, JS, or CSS in page files +- Call GraphQL from partials +- Put raw GraphQL in pages (use `.graphql` files) +- Create or modify application files outside the `app/` directory +- Use more than one HTTP methods per page: +``` +#Never try to handle POST + rendering + redirect in the same root page. Keep it clean: +/ → GET → renders page +/contact (or similar) → POST → processes + redirects +``` +- Set main page methos as POST - it is not PHP! + +--- + +## 25. Pre-Flight Checklist + +Before every change, verify: + +- [ ] No underscore prefix in partial filenames +- [ ] `render 'path/name'` maps to `app/views/partials/path/name.liquid` +- [ ] Pages have ONE HTTP method each +- [ ] No raw GraphQL in pages (use `{% graphql %}` tag with `.graphql` files) +- [ ] No HTML/JS/CSS in pages +- [ ] No hardcoded text in partials (use translations) +- [ ] `platformos-check` passes with 0 errors +- [ ] Every file synced after modification +- [ ] All list queries support pagination (`per_page`, `page`) +- [ ] All inputs validated in commands before persisting +- [ ] CSS/JS minified, `asset_url` used for cache busting + +### Asset URL Usage + +```liquid +{{ 'images/img.png' | asset_url }} +``` diff --git a/src/data/shopify-objects.json b/src/data/shopify-objects.json new file mode 100644 index 0000000..ca93ce3 --- /dev/null +++ b/src/data/shopify-objects.json @@ -0,0 +1,95 @@ +{ + "objects": [ + "product", + "collection", + "cart", + "shop", + "customer", + "order", + "variant", + "line_item", + "theme", + "template", + "section", + "block", + "paginate", + "search", + "article", + "blog", + "comment", + "all_products", + "collections", + "linklists", + "handle", + "canonical_url", + "page_title", + "page_description", + "content_for_header", + "current_page", + "current_tags", + "powered_by_link", + "scripts", + "settings", + "request", + "routes", + "localization", + "gift_card", + "checkout", + "recommendations", + "predictive_search", + "content_for_index", + "shipping_method", + "policy" + ], + "filters": [ + "money", + "money_with_currency", + "money_without_trailing_zeros", + "money_without_currency", + "img_tag", + "img_url", + "asset_img_url", + "shopify_asset_url", + "global_asset_url", + "image_url", + "image_tag", + "product_img_url", + "collection_img_url", + "article_img_url", + "link_to", + "link_to_tag", + "link_to_vendor", + "link_to_type", + "link_to_add_tag", + "link_to_remove_tag", + "payment_type_img_url", + "payment_type_svg_tag", + "within", + "url_for_type", + "url_for_vendor", + "sort_by", + "highlight", + "weight", + "weight_with_unit", + "format_address", + "color_to_rgb", + "color_to_hsl", + "color_modify", + "color_lighten", + "color_darken", + "color_saturate", + "color_desaturate", + "color_mix", + "color_contrast", + "font_face", + "font_modify", + "font_url", + "external_video_tag", + "external_video_url", + "video_tag", + "metafield_tag", + "metafield_text", + "placeholder_svg_tag", + "customer_login_link" + ] +} diff --git a/src/http-server.js b/src/http-server.js index a62c3e0..7ec4a8e 100644 --- a/src/http-server.js +++ b/src/http-server.js @@ -1,18 +1,32 @@ import { createServer } from 'node:http'; -import { readFileSync, readdirSync, existsSync } from 'node:fs'; -import { join } from 'node:path'; +import { readFileSync, writeFileSync, readdirSync, existsSync, mkdirSync, statSync } from 'node:fs'; +import { join, resolve, relative, isAbsolute, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; import { spawn } from 'node:child_process'; import yaml from 'js-yaml'; import { getToolList, dispatchTool } from './tools.js'; import { ToolError } from './core/tool-error.js'; import { HTTP_MAX_BODY } from './core/constants.js'; import { buildDashboardHtml } from './dashboard.js'; +import { getProjectMap } from './tools/project-map.js'; +import { buildDependencyGraph } from './core/dependency-graph.js'; +import { checkScorecards, sessionSummaries, recommendations, toolSequenceBigrams, diagnosticJourney, confidenceCalibration, fixAdoptionFunnel, knowledgeGaps, ruleScoresByCategory, ruleDrilldown, rulePerformance, adaptiveModeImpact, fixRulePerformance } from './core/analytics-queries.js'; +import { ruleScores, suggestedRules, retrieveCasesByCheck, generateRuleTemplate, synthesizeGuardPredicate } from './core/case-base.js'; +import { withCheckLabels, withRuleLabels } from './core/analytics-labels.js'; +import { addPromotedRule, removePromotedRule, listPromotedRules } from './core/rules/promoted-rules.js'; +import { reloadRules, loadAllRules } from './core/rules/index.js'; +import { runRules, getDisabledRules, getAllChecksWithRules, getRulesForCheck, getDisabledRuleDetails, getForceEnabledRules, getForceDisabledRules } from './core/rules/engine.js'; +import { loadOverrides, addForceEnable, addForceDisable, removeOverride } from './core/rule-overrides.js'; +import { loadCacConfig, updateCacConfig, defaultCacConfig, VALID_MODES, VALID_ACTIONS } from './core/cac-config.js'; +import { getRecentCacDecisions } from './core/cac-predictor.js'; +import { extractParams, templateOf, KNOWN_EXTRACTOR_CHECKS } from './core/diagnostic-record.js'; +import { buildFactGraph } from './core/project-fact-graph.js'; /** * HTTP server — REST endpoints for tool discovery, execution, and resources. * MCP protocol (JSON-RPC over stdio) is handled by the SDK transport in server.js. */ -export function startHttp(registry, { port, log, version, logPath, getStatus, restartLsp, dataRoot, subscribeToEvents, posCliPath, projectDir }) { +export function startHttp(registry, { port, log, version, logPath, getStatus, restartLsp, dataRoot, subscribeToEvents, posCliPath, projectDir, sessionsDir, saveSessionSummary, analyticsStore, blobStore, onAnalyticsRebuild, onOverridesChanged, onCacConfigChanged, switchEngineMode, getEngineMode }) { if (!port) return null; const dashboardHtml = buildDashboardHtml(); @@ -35,6 +49,11 @@ export function startHttp(registry, { port, log, version, logPath, getStatus, re return res.end(dashboardHtml); } + // ── Vendor static files ───────────────────────────────────────────── + if (method === 'GET' && url.pathname.startsWith('/vendor/')) { + return handleVendorFile(url.pathname, res); + } + // ── GET routes ────────────────────────────────────────────────────── if (method === 'GET' && url.pathname === '/health') { return sendJson(res, 200, { status: 'ok', server: 'pos-supervisor', version }); @@ -70,8 +89,51 @@ export function startHttp(registry, { port, log, version, logPath, getStatus, re return handleGetEnvs(projectDir, res); } - // ── POST routes (need body parsing) ───────────────────────────────── + if (method === 'GET' && url.pathname === '/api/suppressions') { + return handleGetSuppressions(projectDir, res); + } + + if (method === 'GET' && url.pathname === '/api/engine/mode') { + return sendJson(res, 200, { mode: getEngineMode?.() ?? 'static' }); + } + + if (method === 'GET' && url.pathname === '/api/sessions') { + return handleGetSessions(sessionsDir, res); + } + + if (method === 'GET' && url.pathname === '/api/file') { + return handleGetFile(projectDir, url, res); + } + + if (method === 'GET' && url.pathname === '/api/dependency-tree') { + return handleGetDependencyTree(projectDir, getStatus, res); + } + + if (method === 'GET' && url.pathname === '/api/rules/promoted') { + return handleGetPromotedRules(projectDir, res); + } + + // ── DELETE routes ────────────────────────────────────────────────────── + if (method === 'DELETE' && url.pathname === '/api/rules/promote') { + return handleDeletePromotedRule(projectDir, url, res); + } + + // ── POST routes (no body) ──────────────────────────────────────────── if (method === 'POST') { + if (url.pathname === '/api/lsp/restart') { + return handleLspRestart(restartLsp, res); + } + + if (url.pathname === '/api/sessions/save') { + if (saveSessionSummary) { saveSessionSummary(); } + return sendJson(res, 200, { ok: true }); + } + + if (url.pathname === '/api/analytics/rebuild') { + return handleAnalyticsRebuild(analyticsStore, sessionsDir, onAnalyticsRebuild, res); + } + + // ── POST routes (need body parsing) ─────────────────────────────── let body; try { body = await readJsonBody(req); @@ -83,8 +145,8 @@ export function startHttp(registry, { port, log, version, logPath, getStatus, re return handleCall(registry, body, res); } - if (url.pathname === '/api/lsp/restart') { - return handleLspRestart(restartLsp, res); + if (url.pathname === '/api/analytics/baseline') { + return handleAnalyticsBaselineSet(analyticsStore, body, res); } if (url.pathname === '/api/pos-cli/data-clean') { @@ -94,8 +156,141 @@ export function startHttp(registry, { port, log, version, logPath, getStatus, re if (url.pathname === '/api/pos-cli/deploy') { return handlePosCliCommand(posCliPath, projectDir, body, 'deploy', log, res); } + + if (url.pathname === '/api/rules/promote') { + return handlePromoteRule(projectDir, body, res); + } + + if (url.pathname === '/api/engine/mode') { + return handleSetEngineMode(switchEngineMode, body, log, res); + } + + if (url.pathname === '/api/health-score') { + return handlePostHealthScore(analyticsStore, body, res); + } + + if (url.pathname === '/api/suppressions') { + return handlePostSuppression(projectDir, body, log, res); + } + + if (url.pathname === '/api/rules/test') { + return handleRuleTest(body, res, analyticsStore, projectDir); + } + + if (url.pathname === '/api/engine/rule-overrides') { + return handleRuleOverridesMutate(projectDir, body, res, log, onOverridesChanged); + } + + if (url.pathname === '/api/cac/config') { + return handleCacConfigMutate(projectDir, body, res, log, onCacConfigChanged); + } + } + + // ── Analytics GET routes ────────────────────────────────────────────── + if (method === 'GET' && url.pathname === '/api/analytics/stats') { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + return sendJson(res, 200, analyticsStore.stats()); + } + + if (method === 'GET' && url.pathname === '/api/analytics/baseline') { + return handleAnalyticsBaselineGet(analyticsStore, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/scorecards') { + return handleAnalyticsScorecards(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/sessions') { + return handleAnalyticsSessions(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/recommendations') { + return handleAnalyticsRecommendations(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/bigrams') { + return handleAnalyticsBigrams(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/rule-scores') { + return handleRuleScores(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/rule-performance') { + return handleRulePerformance(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/fix-rule-performance') { + return handleFixRulePerformance(analyticsStore, url, res); } + if (method === 'GET' && url.pathname === '/api/analytics/rule-drilldown') { + return handleRuleDrilldown(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/suggested-rules') { + return handleSuggestedRules(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/cases') { + return handleCases(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/health-scores') { + return handleGetHealthScores(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/journey') { + return handleDiagnosticJourney(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/calibration') { + return handleConfidenceCalibration(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/funnel') { + return handleFixAdoptionFunnel(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/knowledge-gaps') { + return handleKnowledgeGaps(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/analytics/rule-heatmap') { + return handleRuleHeatmap(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/rules/checks') { + return handleRuleChecks(res); + } + + if (method === 'GET' && url.pathname === '/api/engine-map') { + return handleEngineMap(analyticsStore, res); + } + + if (method === 'GET' && url.pathname === '/api/blob') { + return handleBlobRead(blobStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/engine/impact') { + return handleEngineImpact(analyticsStore, url, res); + } + + if (method === 'GET' && url.pathname === '/api/engine/rule-overrides') { + return handleRuleOverridesList(projectDir, res, log); + } + // POST on this path is dispatched inside the POST block above so the + // shared body-parser isn't read twice. + + if (method === 'GET' && url.pathname === '/api/cac/config') { + return handleCacConfigGet(projectDir, res, log); + } + + if (method === 'GET' && url.pathname === '/api/cac/decisions') { + return handleCacDecisions(url, res); + } + // POST /api/cac/config is dispatched inside the POST block above. + // ── Fallback ──────────────────────────────────────────────────────── sendJson(res, 404, { error: 'Not found' }); }); @@ -219,19 +414,127 @@ function handleGetKnowledge(dataRoot, res) { } } +/** + * Two coexisting hint subsystems are joined here: + * • static — `src/data/hints/.md` rendered into the diagnostic by + * error-enricher.js. Legacy LSP checks; one fixed blob each. + * • rule — `src/core/rules/.js` builds the hint dynamically per + * diagnostic. No md file exists; the registry is the source. + * + * Pre-fix the endpoint only knew about (1) and 404'd on every (2). The + * dashboard drilldown silently broke for the 12+ rule-driven checks + * (GraphQLVariablesCheck, PartialCallArguments, NonGetRenderingPage, …). + * + * Response shape: + * GET /api/hints + * { hints: [name, …], // backward-compat: union of both kinds + * checks: [{ name, sources: ['static'|'rule', …] }, …] } + * GET /api/hints?name= + * { name, content, source: 'static' } // md found + * { name, content, source: 'rule', rule_ids: [...] } // synthesized from registry + * 404 only when both lookups miss. + */ function handleGetHints(dataRoot, url, res) { if (!dataRoot) return sendJson(res, 503, { error: 'Data dir not available' }); const hintsDir = join(dataRoot, 'hints'); const name = url.searchParams.get('name'); + + // Populate the rule registry once. Idempotent — guarded by `_loaded`. + loadAllRules(); + + if (name) { + const file = join(hintsDir, `${name}.md`); + if (existsSync(file)) { + try { + const content = readFileSync(file, 'utf-8'); + return sendJson(res, 200, { name, content, source: 'static' }); + } catch (e) { + // Fall through — let the rule registry resolve it if possible. + } + } + const rules = getRulesForCheck(name); + if (rules.length > 0) { + return sendJson(res, 200, { + name, + content: synthesizeRuleHintDoc(name, rules), + source: 'rule', + rule_ids: rules.map(r => r.id), + }); + } + return sendJson(res, 404, { error: `No hint or rule for ${name}` }); + } + + let staticNames = []; try { - if (name) { - const content = readFileSync(join(hintsDir, `${name}.md`), 'utf-8'); - return sendJson(res, 200, { name, content }); + staticNames = readdirSync(hintsDir).filter(f => f.endsWith('.md')).map(f => f.replace('.md', '')); + } catch { + // hints dir may be missing on a fresh checkout — still return rule names. + } + const ruleNames = getAllChecksWithRules(); + const staticSet = new Set(staticNames); + const ruleSet = new Set(ruleNames); + const all = Array.from(new Set([...staticNames, ...ruleNames])).sort(); + const checks = all.map(n => { + const sources = []; + if (staticSet.has(n)) sources.push('static'); + if (ruleSet.has(n)) sources.push('rule'); + return { name: n, sources }; + }); + sendJson(res, 200, { hints: all, checks }); +} + +/** + * Render a markdown reference doc for a rule-driven check by introspecting + * the registry. Lists each sub-rule with its priority and the source of its + * `when()` predicate (truncated). Surfaces the file path the developer must + * edit to change the hint at runtime. + */ +function synthesizeRuleHintDoc(name, rules) { + const sorted = [...rules].sort((a, b) => a.priority - b.priority); + const moduleBase = name.replace(/^pos-supervisor:/, ''); + const lines = []; + lines.push(`# ${name}`); + lines.push(''); + lines.push( + `*Rule-driven check.* Hints are generated dynamically by ` + + `\`src/core/rules/${moduleBase}.js\` at validation time. There is no static ` + + `\`.md\` for this check — agents see whatever \`apply()\` returns from the ` + + `first matching sub-rule below.` + ); + lines.push(''); + lines.push(`## Sub-rules (${sorted.length})`); + lines.push(''); + lines.push('Engine returns the first match in priority order (lower = higher priority).'); + lines.push(''); + for (const r of sorted) { + lines.push(`### \`${r.id}\` — priority ${r.priority}`); + lines.push(''); + const whenSrc = stringifyRulePredicate(r.when); + if (whenSrc) { + lines.push('```js'); + lines.push(`when: ${whenSrc}`); + lines.push('```'); + lines.push(''); } - const files = readdirSync(hintsDir).filter(f => f.endsWith('.md')).map(f => f.replace('.md', '')); - sendJson(res, 200, { hints: files }); - } catch (e) { - sendJson(res, 404, { error: e.message }); + } + lines.push('---'); + lines.push(''); + lines.push( + `To change the hint shown to agents, edit the relevant \`apply()\` in ` + + `\`src/core/rules/${moduleBase}.js\`. Each \`apply()\` returns ` + + `\`{ rule_id, hint_md, fixes, confidence, see_also? }\` which the validator ` + + `embeds into the diagnostic.` + ); + return lines.join('\n'); +} + +function stringifyRulePredicate(fn) { + if (typeof fn !== 'function') return null; + try { + const src = fn.toString(); + return src.length > 240 ? `${src.slice(0, 237)}...` : src; + } catch { + return null; } } @@ -266,6 +569,771 @@ function handleSse(subscribeToEvents, req, res) { }); } +// ── Suppression file (A3 — false positive manager) ─────────────────────── + +const SUPPRESS_FILE = '.pos-supervisor-ignore.yml'; + +function handleGetSuppressions(projectDir, res) { + if (!projectDir) return sendJson(res, 503, { error: 'Project directory not configured' }); + const filePath = join(projectDir, SUPPRESS_FILE); + if (!existsSync(filePath)) return sendJson(res, 200, { suppressions: [] }); + try { + const content = readFileSync(filePath, 'utf-8'); + const parsed = yaml.load(content); + sendJson(res, 200, { suppressions: parsed?.suppressions || [] }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handlePostSuppression(projectDir, body, log, res) { + if (!projectDir) return sendJson(res, 503, { error: 'Project directory not configured' }); + const { check, file_pattern, reason, action } = body; + + if (!check || typeof check !== 'string') { + return sendJson(res, 400, { error: 'Missing or invalid "check" field' }); + } + + const filePath = join(projectDir, SUPPRESS_FILE); + let existing = { suppressions: [] }; + if (existsSync(filePath)) { + try { + existing = yaml.load(readFileSync(filePath, 'utf-8')) || { suppressions: [] }; + } catch { existing = { suppressions: [] }; } + } + if (!existing.suppressions) existing.suppressions = []; + + if (action === 'remove') { + existing.suppressions = existing.suppressions.filter(s => + !(s.check === check && (s.file_pattern || '') === (file_pattern || '')) + ); + } else { + const dup = existing.suppressions.find(s => + s.check === check && (s.file_pattern || '') === (file_pattern || '') + ); + if (!dup) { + const entry = { check }; + if (file_pattern) entry.file_pattern = file_pattern; + if (reason) entry.reason = reason; + entry.added_at = new Date().toISOString(); + existing.suppressions.push(entry); + } + } + + try { + writeFileSync(filePath, yaml.dump(existing, { lineWidth: 120 })); + log?.(`Suppression ${action === 'remove' ? 'removed' : 'added'}: ${check}${file_pattern ? ' (' + file_pattern + ')' : ''}`); + sendJson(res, 200, { ok: true, suppressions: existing.suppressions }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── Session history (D3 — comparative session view) ────────────────────── + +function handleGetSessions(sessionsDir, res) { + if (!sessionsDir) return sendJson(res, 200, { sessions: [] }); + if (!existsSync(sessionsDir)) return sendJson(res, 200, { sessions: [] }); + try { + const files = readdirSync(sessionsDir).filter(f => f.endsWith('.json')).sort().reverse(); + const sessions = files.slice(0, 50).map(f => { + try { return JSON.parse(readFileSync(join(sessionsDir, f), 'utf-8')); } + catch { return null; } + }).filter(Boolean); + sendJson(res, 200, { sessions }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── Project file reader (D1 — live diagnostic console) ────────────────── + +const FILE_READ_MAX_BYTES = 512 * 1024; +const FILE_READ_EXTS = new Set(['.liquid', '.graphql', '.yml', '.yaml', '.md', '.html', '.css', '.js', '.json']); + +function handleGetFile(projectDir, url, res) { + if (!projectDir) return sendJson(res, 503, { error: 'Project directory not configured' }); + const rel = url.searchParams.get('path'); + if (!rel || typeof rel !== 'string') return sendJson(res, 400, { error: 'Missing path parameter' }); + if (isAbsolute(rel)) return sendJson(res, 400, { error: 'Path must be relative to project root' }); + + const projectRoot = resolve(projectDir); + const target = resolve(projectRoot, rel); + const relCheck = relative(projectRoot, target); + if (relCheck.startsWith('..') || isAbsolute(relCheck)) { + return sendJson(res, 403, { error: 'Path escapes project root' }); + } + + const dotIdx = target.lastIndexOf('.'); + const ext = dotIdx >= 0 ? target.slice(dotIdx).toLowerCase() : ''; + if (!FILE_READ_EXTS.has(ext)) { + return sendJson(res, 415, { error: 'File extension not allowed for preview: ' + ext }); + } + + if (!existsSync(target)) return sendJson(res, 404, { error: 'File not found' }); + + try { + const st = statSync(target); + if (!st.isFile()) return sendJson(res, 400, { error: 'Not a regular file' }); + if (st.size > FILE_READ_MAX_BYTES) { + return sendJson(res, 413, { error: 'File too large (max 512 KB)' }); + } + const content = readFileSync(target, 'utf-8'); + sendJson(res, 200, { path: rel, ext, content }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── Dependency impact tree ─────────────────────────────────────────────── + +async function handleGetDependencyTree(projectDir, getStatus, res) { + if (!projectDir) return sendJson(res, 503, { error: 'Project directory not configured' }); + try { + const projectMap = await getProjectMap(projectDir); + const graph = buildDependencyGraph(projectMap); + + const fileHistory = getStatus ? (getStatus()?.fileHistory || []) : []; + const stateByPath = Object.create(null); + for (const f of fileHistory) { + const errors = f.lastErrorCount || 0; + const warnings = f.lastWarningCount || 0; + const state = errors > 0 ? 'dirty' + : warnings > 0 ? 'warned' + : (f.calls || 0) > 1 ? 'fixed' + : 'clean'; + stateByPath[f.path] = { state, calls: f.calls || 0, errors, warnings, streak: f.consecutiveNonDecreasing || 0 }; + } + + const nodes = {}; + for (const [path, edges] of Object.entries(graph)) { + nodes[path] = { + depends_on: edges.depends_on, + referenced_by: edges.referenced_by, + validation: stateByPath[path] || null, + }; + } + + sendJson(res, 200, { nodes, total: Object.keys(nodes).length, generated_at: new Date().toISOString() }); + } catch (err) { + sendJson(res, 500, { error: err.message }); + } +} + +// ── Engine mode handler ───────────────────────────────────────────────────── + +function handleSetEngineMode(switchEngineMode, body, log, res) { + if (!switchEngineMode) return sendJson(res, 503, { error: 'Engine mode switching not available' }); + const { mode } = body ?? {}; + if (!mode || (mode !== 'adaptive' && mode !== 'static')) { + return sendJson(res, 400, { error: 'Invalid mode. Must be "adaptive" or "static".' }); + } + try { + const newMode = switchEngineMode(mode); + log?.(`engine-mode: switched to ${newMode} via HTTP`); + sendJson(res, 200, { mode: newMode }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── Promoted rules handlers (Phase J) ───────────────────────────────────── + +function handleGetPromotedRules(projectDir, res) { + if (!projectDir) return sendJson(res, 503, { error: 'Project directory not configured' }); + try { + const rules = listPromotedRules(projectDir); + sendJson(res, 200, { rules }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handlePromoteRule(projectDir, body, res) { + if (!projectDir) return sendJson(res, 503, { error: 'Project directory not configured' }); + + const { id, check, priority, when, apply } = body; + if (!id || typeof id !== 'string') return sendJson(res, 400, { error: 'Missing or invalid "id" field' }); + if (!check || typeof check !== 'string') return sendJson(res, 400, { error: 'Missing or invalid "check" field' }); + if (!apply?.hint_md) return sendJson(res, 400, { error: 'Missing "apply.hint_md" field' }); + + const entry = { + id, + check, + priority: priority ?? 55, + origin: 'promoted', + promoted_at: new Date().toISOString(), + probation: true, + when: when ?? {}, + apply, + }; + + try { + const supervisorDir = join(projectDir, '.pos-supervisor'); + if (!existsSync(supervisorDir)) mkdirSync(supervisorDir, { recursive: true }); + addPromotedRule(projectDir, entry); + reloadRules(projectDir); + sendJson(res, 201, { ok: true, rule: entry }); + } catch (e) { + const status = e.message.includes('already exists') ? 409 : 500; + sendJson(res, status, { error: e.message }); + } +} + +function handleDeletePromotedRule(projectDir, url, res) { + if (!projectDir) return sendJson(res, 503, { error: 'Project directory not configured' }); + const ruleId = url.searchParams.get('id'); + if (!ruleId) return sendJson(res, 400, { error: 'Missing "id" query parameter' }); + + try { + removePromotedRule(projectDir, ruleId); + reloadRules(projectDir); + sendJson(res, 200, { ok: true, removed: ruleId }); + } catch (e) { + const status = e.message.includes('not found') ? 404 : 500; + sendJson(res, status, { error: e.message }); + } +} + +// ── Analytics query handlers (Phase K2-K5) ──────────────────────────────── + +function handleDiagnosticJourney(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + let templateFp = url.searchParams.get('template_fp'); + const check = url.searchParams.get('check'); + if (!templateFp && check) { + const row = analyticsStore.queryOne( + `SELECT template_fp, COUNT(*) as cnt FROM diagnostics WHERE check_name = ? AND template_fp IS NOT NULL GROUP BY template_fp ORDER BY cnt DESC LIMIT 1`, + [check], + ); + templateFp = row?.template_fp; + } + if (!templateFp) return sendJson(res, 400, { error: 'template_fp or check parameter required' }); + const journey = diagnosticJourney(analyticsStore, templateFp, { since }); + sendJson(res, 200, { ...journey, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleConfidenceCalibration(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const buckets = parseInt(url.searchParams.get('buckets') || '10', 10); + const calibration = confidenceCalibration(analyticsStore, { + buckets: Math.min(Math.max(buckets, 2), 20), + since, + }); + sendJson(res, 200, { calibration, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleFixAdoptionFunnel(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const funnel = fixAdoptionFunnel(analyticsStore, { since }); + sendJson(res, 200, { ...funnel, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleKnowledgeGaps(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const gaps = knowledgeGaps(analyticsStore, { since }); + sendJson(res, 200, { gaps, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleRuleHeatmap(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const cells = ruleScoresByCategory(analyticsStore, { since }); + sendJson(res, 200, { cells, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +const CHECK_EXAMPLES = { + UnknownFilter: 'Unknown filter "to_json"', + UndefinedObject: "Variable 'product' is undefined", + UnusedAssign: "The variable 'x' is assigned but not used", + MissingPartial: "'forms/login' does not exist", + TranslationKeyExists: "Translation key 'a.b.c' not found. Did you mean 'a.b.cd'?", + UnknownProperty: "Unknown property `name` on `current_user`", + MissingRenderPartialArguments: "Missing required argument 'email' in render tag for partial 'sessions/form'", + MetadataParamsCheck: 'Required parameter clear must be passed to function call', + GraphQLCheck: 'Variable "$id" is never used in operation "x"', + DeprecatedTag: "Tag 'include' is deprecated, use 'render'", +}; + +function handleRuleChecks(res) { + try { + loadAllRules(); + const checks = getAllChecksWithRules(); + const result = checks.map(check => { + const rules = getRulesForCheck(check); + return { + check, + rule_count: rules.length, + rule_ids: rules.map(r => r.id), + has_extractor: KNOWN_EXTRACTOR_CHECKS.includes(check), + example_message: CHECK_EXAMPLES[check] || null, + }; + }).sort((a, b) => a.check.localeCompare(b.check)); + sendJson(res, 200, { checks: result }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +async function handleRuleTest(body, res, analyticsStore, projDir) { + try { + const { check, message, file } = body; + if (!check || !message) { + return sendJson(res, 400, { error: 'Missing required fields: check, message' }); + } + + loadAllRules(); + const params = extractParams(check, message); + const tmplFp = templateOf(check, message); + const diag = { check, params, message, file: file || 'app/views/pages/test.liquid', line: 1, template_fp: tmplFp }; + + let graph = null; + let graphAvailable = false; + try { + if (projDir) { + const projectMap = await getProjectMap(projDir); + if (projectMap) { + graph = buildFactGraph(projectMap); + graphAvailable = true; + } + } + } catch { /* project map unavailable — run without graph */ } + + const facts = { graph, filtersIndex: null, objectsIndex: null, tagsIndex: null, schemaIndex: null, analyticsStore }; + + const matched = runRules(diag, facts); + const allMatches = runRules(diag, facts, { multiMatch: true }); + const disabledRules = [...getDisabledRules()]; + + const candidates = getRulesForCheck(check); + const ruleEval = candidates.map(rule => { + if (disabledRules.includes(rule.id)) return { rule_id: rule.id, status: 'disabled' }; + try { + const whenResult = rule.when(diag, facts); + if (!whenResult) return { rule_id: rule.id, status: 'guard_failed' }; + const applyResult = rule.apply(diag, facts); + if (!applyResult) return { rule_id: rule.id, status: 'apply_returned_null' }; + return { rule_id: rule.id, status: 'matched' }; + } catch (e) { + return { rule_id: rule.id, status: 'error', error: e.message }; + } + }); + + const CHECKS_NEEDING_INDEXES = ['UnknownFilter', 'UnknownProperty']; + const notes = []; + if (!graphAvailable) notes.push('Project map unavailable — rules requiring graph data cannot fire.'); + if (!facts.filtersIndex && CHECKS_NEEDING_INDEXES.includes(check)) notes.push('LSP indexes (filters, objects, tags) unavailable — some rules skipped.'); + + sendJson(res, 200, { + input: { check, message, file: diag.file }, + extracted_params: params, + template_fp: tmplFp, + graph_available: graphAvailable, + matched_rule: matched ? { + rule_id: matched.rule_id, + hint_md: matched.hint_md, + confidence: matched.confidence, + fixes: matched.fixes || [], + see_also: matched.see_also || null, + } : null, + all_matches: (allMatches || []).map(r => ({ + rule_id: r.rule_id, + hint_md: r.hint_md, + confidence: r.confidence, + })), + rule_evaluation: ruleEval, + disabled_rules: disabledRules, + note: notes.length > 0 ? notes.join(' ') : null, + }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── Health score handlers (Phase K1) ────────────────────────────────────── + +function handlePostHealthScore(analyticsStore, body, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + const { score, mode, dimensions } = body; + if (typeof score !== 'number' || score < 0 || score > 100) { + return sendJson(res, 400, { error: 'Invalid score — must be a number 0-100' }); + } + if (!mode || typeof mode !== 'string') { + return sendJson(res, 400, { error: 'Missing or invalid "mode" field' }); + } + try { + analyticsStore.insertHealthScore({ score, mode, dimensions: dimensions ?? {} }); + sendJson(res, 201, { ok: true }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handleGetHealthScores(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const limit = Math.min(parseInt(url.searchParams.get('limit') || '30', 10), 200); + const mode = url.searchParams.get('mode') || undefined; + const scores = analyticsStore.getHealthScores({ limit, mode }); + sendJson(res, 200, { scores }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── Vendor static files ────────────────────────────────────────────────── + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const VENDOR_DIR = join(__dirname, 'vendor'); +const VENDOR_MIME = { '.js': 'application/javascript', '.css': 'text/css', '.json': 'application/json' }; + +function handleVendorFile(pathname, res) { + const filename = pathname.replace('/vendor/', ''); + if (filename.includes('..') || filename.includes('/')) { + return sendJson(res, 403, { error: 'Forbidden' }); + } + const filePath = join(VENDOR_DIR, filename); + if (!existsSync(filePath)) return sendJson(res, 404, { error: 'Not found' }); + try { + const content = readFileSync(filePath); + const ext = filename.slice(filename.lastIndexOf('.')); + res.writeHead(200, { + 'Content-Type': VENDOR_MIME[ext] || 'application/octet-stream', + 'Content-Length': content.length, + 'Cache-Control': 'public, max-age=86400', + }); + res.end(content); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── Engine Map ────────────────────────────────────────────────────────── + +const RULE_DEPS = { + 'MissingPartial.module_path': { needs: ['params'], graph_queries: [] }, + 'MissingPartial.file_exists': { needs: ['params', 'graph'], graph_queries: ['hasNode'] }, + 'MissingPartial.suggest_nearest': { needs: ['params', 'graph'], graph_queries: ['nodesByType', 'dependsOn', 'nodeByPath'] }, + 'MissingPartial.create_file': { needs: ['params', 'graph'], graph_queries: ['hasNode'] }, + 'UndefinedObject.shopify_object': { needs: ['params'], graph_queries: [] }, + 'UndefinedObject.context_prefix': { needs: ['params'], graph_queries: [] }, + 'UndefinedObject.declare_param': { needs: ['params'], graph_queries: [] }, + 'UndefinedObject.generic': { needs: ['params'], graph_queries: [] }, + 'UnknownFilter.tag_confusion': { needs: ['params', 'tagsIndex'], graph_queries: [] }, + 'UnknownFilter.shopify_filter': { needs: ['params'], graph_queries: [] }, + 'UnknownFilter.suggest_nearest': { needs: ['params', 'filtersIndex'], graph_queries: [] }, + 'UnknownFilter.generic': { needs: ['params'], graph_queries: [] }, + 'TranslationKeyExists.suggest_nearest': { needs: ['params', 'graph'], graph_queries: ['nodesByType'] }, + 'TranslationKeyExists.create_key': { needs: ['params'], graph_queries: [] }, + 'UnusedAssign.passed_to_render': { needs: ['params', 'graph'], graph_queries: ['renderCallsFrom'] }, + 'UnusedAssign.passed_to_function': { needs: ['params', 'graph'], graph_queries: ['nodeByPath'] }, + 'UnusedAssign.generic': { needs: ['params'], graph_queries: [] }, + 'MissingRenderPartialArguments.doc_block_mismatch': { needs: ['params', 'graph'], graph_queries: ['partialSignature'] }, + 'MissingRenderPartialArguments.chain_satisfied': { needs: ['params', 'graph'], graph_queries: ['nodeByPath'] }, + 'MissingRenderPartialArguments.generic': { needs: ['params'], graph_queries: [] }, + 'UnknownProperty.schema_property': { needs: ['params', 'graph'], graph_queries: ['nodesByType'] }, + 'UnknownProperty.context_property': { needs: ['params', 'objectsIndex'], graph_queries: [] }, + 'UnknownProperty.generic': { needs: ['params'], graph_queries: [] }, + 'MetadataParamsCheck.module_contract': { needs: ['params'], graph_queries: [] }, + 'MetadataParamsCheck.doc_block_params': { needs: ['params', 'graph'], graph_queries: ['partialSignature'] }, + 'MetadataParamsCheck.generic': { needs: ['params'], graph_queries: [] }, + 'GraphQLCheck.unknown_field': { needs: ['params', 'graph'], graph_queries: ['nodesByType'] }, + 'GraphQLCheck.unused_variable': { needs: ['params'], graph_queries: [] }, + 'GraphQLCheck.type_mismatch': { needs: ['params'], graph_queries: [] }, + 'GraphQLCheck.generic': { needs: ['params'], graph_queries: [] }, +}; + +function handleEngineMap(analyticsStore, res) { + try { + loadAllRules(); + const checks = getAllChecksWithRules(); + const disabledSet = getDisabledRules(); + + const extractorChecks = [...KNOWN_EXTRACTOR_CHECKS]; + + const hintFiles = []; + const hintsDir = join(__dirname, 'data', 'hints'); + if (existsSync(hintsDir)) { + for (const f of readdirSync(hintsDir)) { + if (f.endsWith('.md')) { + const name = f.replace('.md', ''); + const isVariant = name.includes('-'); + const baseCheck = isVariant ? name.split('-')[0] : name; + hintFiles.push({ file: f, name, base_check: baseCheck, is_variant: isVariant }); + } + } + } + + let scores = []; + if (analyticsStore) { + try { scores = ruleScores(analyticsStore, { minEmitted: 1 }); } catch { /* no data yet */ } + } + const scoreMap = new Map(scores.map(s => [s.rule_id, s])); + + const checkNodes = checks.map(check => { + const rules = getRulesForCheck(check); + const hasExtractor = extractorChecks.includes(check); + const hints = hintFiles.filter(h => h.base_check === check); + + const ruleNodes = rules.map(r => { + const deps = RULE_DEPS[r.id] || { needs: ['params'], graph_queries: [] }; + const score = scoreMap.get(r.id); + return { + id: r.id, + priority: r.priority, + needs: deps.needs, + graph_queries: deps.graph_queries, + disabled: disabledSet.has(r.id), + score: score ? { + emitted: score.emitted, + resolved: score.resolved, + regressed: score.regressed, + resolution_rate: score.resolution_rate, + regression_rate: score.regression_rate, + effectiveness: score.effectiveness, + disabled: score.disabled, + } : null, + }; + }); + + return { + check, + has_extractor: hasExtractor, + example_message: CHECK_EXAMPLES[check] || null, + hints: hints.map(h => h.name), + rules: ruleNodes, + }; + }); + + const pipeline_steps = [ + 'LSP Diagnostics', + 'Structural Warnings', + 'Diagnostic Pipeline (9 steps)', + 'Rule Engine (first-match)', + 'Error Enricher (fallback)', + 'Fix Generator', + 'Scorecard', + ]; + + const coverage = { + checks_with_rules: checks.length, + checks_with_extractors: extractorChecks.length, + total_rules: checks.reduce((n, c) => n + getRulesForCheck(c).length, 0), + total_hints: hintFiles.length, + disabled_rules: disabledSet.size, + rules_needing_graph: Object.values(RULE_DEPS).filter(d => d.needs.includes('graph')).length, + rules_needing_indexes: Object.values(RULE_DEPS).filter(d => d.needs.includes('filtersIndex') || d.needs.includes('objectsIndex') || d.needs.includes('tagsIndex')).length, + rules_params_only: Object.values(RULE_DEPS).filter(d => d.needs.length === 1 && d.needs[0] === 'params').length, + }; + + sendJson(res, 200, { checks: checkNodes, pipeline_steps, coverage, hint_files: hintFiles }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handleBlobRead(blobStore, url, res) { + if (!blobStore) return sendJson(res, 503, { error: 'blob store not available' }); + const hash = url.searchParams.get('hash'); + if (!hash || !/^[0-9a-f]{64}$/i.test(hash)) { + return sendJson(res, 400, { error: 'hash must be a 64-char hex SHA256 string' }); + } + const text = blobStore.getText(hash); + if (text == null) return sendJson(res, 404, { error: 'blob not found' }); + return sendJson(res, 200, { text }); +} + +/** + * GET /api/engine/impact + * + * Returns the adaptive-mode impact summary: what rules are currently + * disabled/promoted/overridden, window-scoped emit counts and the split + * between rules that *would* fire under static mode (currently disabled) + * vs adaptive (currently firing). Payload is a merge of the live engine + * state (not in the DB) and adaptiveModeImpact() (DB-derived window query). + */ +function handleEngineImpact(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const windowMs = parseInt(url.searchParams.get('window_ms') || String(86_400_000), 10); + const impact = adaptiveModeImpact(analyticsStore, { windowMs }); + + const disabled = getDisabledRuleDetails(); + const forceEnabled = [...getForceEnabledRules()]; + const forceDisabled = [...getForceDisabledRules()]; + + // Counterfactual: sum window emits that hit currently-disabled rule_ids. + // These are the diagnostics the operator would have seen under static + // mode. A rule in force_enabled is disabled-by-data but running anyway, + // so it still contributes to the adaptive view — exclude it from the + // suppressed sum. + let suppressed_by_disabled = 0; + const per_rule_suppressed = {}; + for (const row of disabled) { + if (row.force_enabled) continue; + const hits = impact.emits_by_rule[row.rule_id] ?? 0; + if (hits > 0) { + suppressed_by_disabled += hits; + per_rule_suppressed[row.rule_id] = hits; + } + } + + return sendJson(res, 200, { + window: { + ms: impact.window_ms, + start: impact.window_start, + end: impact.window_end, + }, + emits_in_window: impact.emits_in_window, + rule_matched_in_window: impact.rule_matched_in_window, + confidence: impact.confidence, + disabled_rules: disabled, + force_enabled: forceEnabled, + force_disabled: forceDisabled, + counterfactual: { + suppressed_by_disabled, + per_rule_suppressed, + }, + }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handleRuleOverridesList(projectDir, res, log) { + try { + const state = loadOverrides(projectDir, { log }); + sendJson(res, 200, state); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +/** + * POST /api/engine/rule-overrides + * + * Body: `{ action: 'force_enable' | 'force_disable' | 'clear', rule_id: string, reason?: string }`. + * + * clear → removes any override for the rule. The `onOverridesChanged` hook + * re-reads the file into the engine and runs `syncDisabledRules` so the + * effect is visible immediately without restart. + */ +function handleRuleOverridesMutate(projectDir, body, res, log, onOverridesChanged) { + const { action, rule_id, reason } = body ?? {}; + if (!rule_id || typeof rule_id !== 'string') { + return sendJson(res, 400, { error: 'rule_id required' }); + } + try { + let state; + if (action === 'force_enable') state = addForceEnable(projectDir, rule_id, reason ?? '', { log }); + else if (action === 'force_disable') state = addForceDisable(projectDir, rule_id, reason ?? '', { log }); + else if (action === 'clear') state = removeOverride(projectDir, rule_id, { log }); + else return sendJson(res, 400, { error: 'action must be force_enable | force_disable | clear' }); + + try { onOverridesChanged?.(); } catch (e) { log(`onOverridesChanged failed: ${e.message}`); } + sendJson(res, 200, state); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +// ── CAC predictor (Cohen's Agentic Conjecture) ─────────────────────────── +// +// Opt-in 4th gating axis. The validator behaves identically to a build +// without the predictor when `enabled: false`. These endpoints expose the +// persisted config + recent decision telemetry to the dashboard. + +function handleCacConfigGet(projectDir, res, log) { + try { + const state = loadCacConfig(projectDir, { log }); + sendJson(res, 200, { + config: state, + defaults: defaultCacConfig(), + valid_modes: VALID_MODES, + valid_actions: VALID_ACTIONS, + }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +/** + * POST /api/cac/config + * + * Body: any subset of `{ enabled, mode, threshold, action, min_samples }`. + * Unknown keys are dropped; out-of-range values are coerced to defaults. + * The `onCacConfigChanged` hook re-reads the file into the live ref so the + * change takes effect immediately for in-flight validate_code calls + * (without requiring a server restart). + */ +function handleCacConfigMutate(projectDir, body, res, log, onCacConfigChanged) { + if (!body || typeof body !== 'object') { + return sendJson(res, 400, { error: 'body required' }); + } + try { + const state = updateCacConfig(projectDir, body, { log }); + try { onCacConfigChanged?.(); } catch (e) { log?.(`onCacConfigChanged failed: ${e.message}`); } + sendJson(res, 200, { config: state }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handleCacDecisions(url, res) { + const limit = clampInt(url.searchParams.get('limit'), 1, 200, 50); + try { + const decisions = getRecentCacDecisions(limit); + sendJson(res, 200, { + count: decisions.length, + decisions, + summary: summarizeCacDecisions(decisions), + }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function summarizeCacDecisions(decisions) { + const out = { allow: 0, downgrade: 0, suppress: 0, by_feature: {}, by_mode: {} }; + for (const d of decisions) { + const dec = d.decision || 'allow'; + out[dec] = (out[dec] ?? 0) + 1; + out.by_feature[d.feature] = (out.by_feature[d.feature] ?? 0) + 1; + out.by_mode[d.mode] = (out.by_mode[d.mode] ?? 0) + 1; + } + return out; +} + +function clampInt(raw, min, max, fallback) { + const n = parseInt(raw, 10); + if (!Number.isFinite(n)) return fallback; + return Math.max(min, Math.min(max, n)); +} + // ── Helpers ─────────────────────────────────────────────────────────────── function sendJson(res, status, data) { @@ -291,6 +1359,234 @@ function readLogTail(logPath, limit) { } } +// ── Analytics handlers (Phase B) ─────────────────────────────────────────── + +/** + * Parse the `since` query parameter into the value the analytics-queries + + * case-base reporting paths accept: + * + * - `?since=all` → null (explicit bypass — operator clicked + * "All time" in the dashboard) + * - `?since=ISO` → string (explicit override) + * - `?since` absent / empty → undefined (function looks up meta baseline) + * + * Validates the ISO string by attempting Date parse; rejects with a thrown + * Error so the surrounding try/catch returns 400. Strict validation is the + * point — silently accepting garbage means an operator typing a bad date + * sees stats they don't expect with no error. + * + * Exported so unit tests can pin the parsing contract without spinning up + * the HTTP server. (Server startup uses bun:sqlite via analytics-store, + * which fails under integration tests that spawn `node bin/...`.) + */ +export function parseSinceParam(url) { + const raw = url.searchParams.get('since'); + if (raw == null || raw === '') return undefined; + if (raw === 'all') return null; + const parsed = new Date(raw); + if (Number.isNaN(parsed.getTime())) { + throw new Error(`since must be 'all' or a valid ISO timestamp; got '${raw}'`); + } + return raw; +} + +function handleAnalyticsBaselineGet(analyticsStore, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + sendJson(res, 200, analyticsStore.getBaselineMeta()); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handleAnalyticsBaselineSet(analyticsStore, body, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + // Body shape: { baseline_ts: ISO } to set, { baseline_ts: null } to clear. + if (!body || typeof body !== 'object') { + return sendJson(res, 400, { error: 'request body must be an object' }); + } + if (!('baseline_ts' in body)) { + return sendJson(res, 400, { error: 'missing required field: baseline_ts (ISO string or null)' }); + } + analyticsStore.setBaselineTs(body.baseline_ts); + sendJson(res, 200, { ok: true, ...analyticsStore.getBaselineMeta() }); + } catch (e) { + // setBaselineTs throws TypeError on invalid input — surface as 400. + const status = e instanceof TypeError ? 400 : 500; + sendJson(res, status, { error: e.message }); + } +} + +function handleAnalyticsRebuild(analyticsStore, sessionsDir, onAnalyticsRebuild, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + if (!sessionsDir) return sendJson(res, 400, { error: 'sessions dir not configured' }); + try { + const result = analyticsStore.rebuild(sessionsDir); + try { onAnalyticsRebuild?.(); } catch {} + sendJson(res, 200, { ok: true, ...result }); + } catch (e) { + sendJson(res, 500, { error: e.message }); + } +} + +function handleAnalyticsScorecards(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const sessionId = url.searchParams.get('session_id') || undefined; + const minCohort = parseInt(url.searchParams.get('min_cohort') || '10', 10); + const cards = checkScorecards(analyticsStore, { sessionId, minCohort, since }); + sendJson(res, 200, { scorecards: withCheckLabels(cards), since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleAnalyticsSessions(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const summaries = sessionSummaries(analyticsStore, { since }); + sendJson(res, 200, { sessions: summaries, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleAnalyticsRecommendations(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const threshold = parseFloat(url.searchParams.get('threshold') || '0.3'); + const recs = recommendations(analyticsStore, threshold, { since }); + sendJson(res, 200, { recommendations: recs, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleAnalyticsBigrams(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const sessionId = url.searchParams.get('session_id') || undefined; + const bigrams = toolSequenceBigrams(analyticsStore, { sessionId, since }); + sendJson(res, 200, { bigrams, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleRuleScores(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const minEmitted = parseInt(url.searchParams.get('min_emitted') || '5', 10); + const scores = ruleScores(analyticsStore, { minEmitted, since }); + sendJson(res, 200, { scores: withRuleLabels(scores), since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleRulePerformance(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const minEmitted = parseInt(url.searchParams.get('min_emitted') || '1', 10); + const scores = rulePerformance(analyticsStore, { minEmitted, since }); + sendJson(res, 200, { scores: withRuleLabels(scores), since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleFixRulePerformance(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const minProposed = parseInt(url.searchParams.get('min_proposed') || '1', 10); + const scores = fixRulePerformance(analyticsStore, { minProposed, since }); + sendJson(res, 200, { scores, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleRuleDrilldown(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const ruleId = url.searchParams.get('rule_id'); + if (!ruleId) return sendJson(res, 400, { error: 'rule_id parameter required' }); + const limit = Math.min(parseInt(url.searchParams.get('limit') || '30', 10), 100); + const data = ruleDrilldown(analyticsStore, ruleId, { limit, since }); + sendJson(res, 200, { ...data, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleSuggestedRules(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const suggestions = suggestedRules(analyticsStore, new Set(), { since }).map(s => { + // Forward the same since to guard synthesis so the inferred guard + // window matches the suggestion window. + const guards = synthesizeGuardPredicate(analyticsStore, s.check, s.template_fp, { since }); + return { + ...s, + when: guards, + template: generateRuleTemplate(s, guards), + }; + }); + sendJson(res, 200, { suggestions, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +function handleCases(analyticsStore, url, res) { + if (!analyticsStore) return sendJson(res, 503, { error: 'analytics store not available' }); + try { + const since = parseSinceParam(url); + const check = url.searchParams.get('check'); + if (!check) return sendJson(res, 400, { error: 'check parameter required' }); + const cases = retrieveCasesByCheck(analyticsStore, check, { minCases: 1, since }); + sendJson(res, 200, { cases, since: resolvedSinceForResponse(analyticsStore, since) }); + } catch (e) { + sendJson(res, sinceErrorStatus(e), { error: e.message }); + } +} + +/** + * Surface what the queries actually filtered by. When `since` was undefined + * (meta default), this returns the meta value so the dashboard can show a + * "Stats since: " banner without a separate round-trip. When the + * caller explicitly passed `?since=all`, returns null. Tolerates errors so + * a missing baseline meta column never breaks the analytics response. + */ +function resolvedSinceForResponse(store, sinceArg) { + if (sinceArg === null) return null; + if (typeof sinceArg === 'string') return sinceArg; + try { + return store.getBaselineTs?.() ?? null; + } catch { + return null; + } +} + +/** + * `parseSinceParam` throws on a malformed `since` query param; surface as + * 400 (client error). Anything else propagates the existing 500 path. + */ +function sinceErrorStatus(err) { + if (err && typeof err.message === 'string' && err.message.includes("since must be")) return 400; + return 500; +} + function readJsonBody(req) { return new Promise((resolve, reject) => { const chunks = []; diff --git a/src/server.js b/src/server.js index f2c7b01..2e5b421 100644 --- a/src/server.js +++ b/src/server.js @@ -1,5 +1,5 @@ import { realpath } from 'node:fs/promises'; -import { readFileSync } from 'node:fs'; +import { readFileSync, writeFileSync, mkdirSync, existsSync, watch } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; @@ -13,9 +13,19 @@ import { toUri } from './core/utils.js'; import { startFsWatcher } from './core/fs-watcher.js'; import { invalidateProjectMap } from './tools/project-map.js'; import { createToolRegistry } from './tools.js'; +import { initPromotedRules, reloadRules } from './core/rules/index.js'; +import { updateDisabledRules, updateForceOverrides, setDisabledRuleDetails } from './core/rules/engine.js'; +import { ruleScores, resolveProbation } from './core/case-base.js'; +import { loadOverrides, overrideSets } from './core/rule-overrides.js'; +import { loadCacConfig } from './core/cac-config.js'; +import { rehydrateRecentCacDecisions } from './core/cac-predictor.js'; +import { loadEngineMode, isAdaptive, setEngineMode, getEngineMode } from './core/engine-mode.js'; import { startHttp } from './http-server.js'; import { createLogger } from './core/logger.js'; import { LSP_READY_TIMEOUT_MS } from './core/constants.js'; +import { startSessionEventBus } from './core/session-event-bus.js'; +import { openBlobStore } from './core/blob-store.js'; +import { openAnalyticsStore } from './core/analytics-store.js'; const __dirname = dirname(fileURLToPath(import.meta.url)); const pkg = JSON.parse(readFileSync(join(__dirname, '..', 'package.json'), 'utf-8')); @@ -31,6 +41,182 @@ const VERSION = pkg.version; export async function createServer({ projectDir, httpPort = 0 }) { const { emit: rawEmit, log, close: closeLogger, logPath } = createLogger({ directory: projectDir, version: VERSION }); + // ── Session id + event bus (Phase A1) ───────────────────────────────────── + // + // The bus owns the append-only NDJSON log + the in-memory projection. It + // runs in PARALLEL with the legacy session.* state during Phase 2 so the + // existing dashboard/tests keep working unchanged. Phase 3's acceptance + // gate compares projection-from-disk to the live projection; once we + // trust that match the legacy mutation paths come out (separate change). + // + // Bus creation is best-effort: if the writer can't open (read-only fs, + // permission denied), the bus runs in-memory only and the live request + // path is never affected. + const sessionsDir = join(projectDir, '.pos-supervisor', 'sessions'); + const sessionId = `session-${new Date().toISOString().replace(/[:.]/g, '-')}`; + const sessionBus = startSessionEventBus({ sessionId, sessionsDir, log }); + if (sessionBus.writerError) { + log(`session-event-bus: running in-memory only (${sessionBus.writerError})`); + } else { + log(`session-event-bus: writing events to ${sessionBus.eventsPath}`); + } + sessionBus.startInvariantInterval(); + + // ── Content blob store (Phase A4/A5) ────────────────────────────────────── + let blobStore = null; + try { + blobStore = openBlobStore(join(projectDir, '.pos-supervisor', 'blobs')); + } catch (e) { + log(`blob-store: failed to open (${e.message}); content hashes will not be stored`); + } + + // ── Analytics store (Phase B — derived from NDJSON, disposable) ──────────── + let analyticsStore = null; + try { + analyticsStore = openAnalyticsStore( + join(projectDir, '.pos-supervisor', 'analytics.db'), + { blobStore }, + ); + log('analytics-store: opened'); + } catch (e) { + log(`analytics-store: failed to open (${e.message}); analytics will not be available`); + } + + // ── Engine mode (adaptive vs static) ────────────────────────────────────── + const engineMode = loadEngineMode(projectDir); + log(`engine-mode: ${engineMode}`); + + // ── Promoted rules (Phase J — declarative rules from analytics) ────────────── + try { + initPromotedRules(projectDir); + log(`promoted-rules: ${isAdaptive() ? 'loaded' : 'skipped (static mode)'}`); + } catch (e) { + log(`promoted-rules: failed to load (${e.message})`); + } + + let promotedRulesWatcher = null; + const promotedRulesPath = join(projectDir, '.pos-supervisor', 'promoted-rules.json'); + const supervisorDir = join(projectDir, '.pos-supervisor'); + if (existsSync(supervisorDir)) { + try { + let debounceTimer = null; + promotedRulesWatcher = watch(supervisorDir, { recursive: false }, (eventType, filename) => { + if (filename !== 'promoted-rules.json') return; + if (debounceTimer) clearTimeout(debounceTimer); + debounceTimer = setTimeout(() => { + debounceTimer = null; + try { + reloadRules(projectDir); + log('promoted-rules: reloaded after file change'); + } catch (e) { + log(`promoted-rules: reload failed (${e.message})`); + } + }, 200); + if (typeof debounceTimer.unref === 'function') debounceTimer.unref(); + }); + if (typeof promotedRulesWatcher.unref === 'function') promotedRulesWatcher.unref(); + } catch (e) { + log(`promoted-rules watcher: failed to start (${e.message})`); + } + } + + // ── Disabled rule enforcement (Phase J4 + I4 operator overrides) ───────────── + // force-enable wins over case-base disable; force-disable applies always. + // Engine picks up the split via ruleIsActive; sync loads file → engine so + // edits made through the dashboard take effect without restart. + function syncRuleOverrides() { + try { + const state = loadOverrides(projectDir, { log }); + const { force_enable, force_disable } = overrideSets(state); + updateForceOverrides({ force_enable, force_disable }); + if (force_enable.size || force_disable.size) { + log(`rule-overrides: ${force_enable.size} force-enabled, ${force_disable.size} force-disabled`); + } + return state; + } catch (e) { + log(`rule-overrides: sync failed (${e.message})`); + return { force_enable: {}, force_disable: {} }; + } + } + + function syncDisabledRules() { + if (!isAdaptive()) { + updateDisabledRules(null); + setDisabledRuleDetails([]); + return; + } + if (!analyticsStore) return; + try { + // Engine state: NEVER apply the operator's reporting baseline. Auto-disable + // requires full history so a freshly-set baseline can't accidentally + // narrow the sample below the disable threshold and re-enable harmful + // rules. `since: null` is the explicit bypass — see case-base.ruleScores + // JSDoc and the `resolveSince` contract. + const scores = ruleScores(analyticsStore, { minEmitted: 5, since: null }); + const disabled = scores.filter(s => s.disabled).map(s => s.rule_id); + updateDisabledRules(disabled); + setDisabledRuleDetails(scores.filter(s => s.disabled)); + if (disabled.length > 0) log(`disabled-rules: ${disabled.length} rule(s) disabled by analytics`); + } catch (e) { + log(`disabled-rules: sync failed (${e.message})`); + } + } + + // Order matters: overrides first so the disabled-rules sync below sees + // them in effect. Both are idempotent — safe to call repeatedly. + syncRuleOverrides(); + syncDisabledRules(); + + // ── CAC predictor config (opt-in 4th gating axis) ────────────────────────── + // Shared mutable ref: validate-code reads `current` on each call, the HTTP + // POST handler mutates it after persisting to disk. Disabled by default — + // when `enabled: false`, validate-code skips the predictor entirely. + const cacConfigState = { current: loadCacConfig(projectDir, { log }) }; + function syncCacConfig() { + try { + cacConfigState.current = loadCacConfig(projectDir, { log }); + const c = cacConfigState.current; + if (c.enabled) { + log(`cac-predictor: ${c.mode} mode, threshold=${c.threshold}, action=${c.action}, min_samples=${c.min_samples}`); + } + } catch (e) { + log(`cac-predictor: sync failed (${e.message})`); + } + } + syncCacConfig(); + + // Rehydrate the CAC decision ring from prior sessions' NDJSON logs so the + // dashboard's "Recent CAC Decisions" panel survives server restarts. Pure + // disk read — runs even when the predictor is disabled, since flipping it + // on later in the session shouldn't show an empty audit trail. Best + // effort: any I/O error returns 0 and is logged at info level. + try { + const n = rehydrateRecentCacDecisions(sessionsDir); + if (n > 0) log(`cac-predictor: rehydrated ${n} decision(s) from prior sessions`); + } catch (e) { + log(`cac-predictor: rehydration failed (${e.message})`); + } + + // ── Engine mode transitions ────────────────────────────────────────────────── + function handleModeTransition(prev, mode) { + log(`engine-mode: ${prev} → ${mode}`); + reloadRules(projectDir); + if (mode === 'adaptive') { + syncDisabledRules(); + if (analyticsStore) { + try { resolveProbation(analyticsStore); } catch {} + } + } else { + updateDisabledRules(null); + } + broadcastSse({ event: 'engine_mode_changed', ts: new Date().toISOString(), prev, mode }); + } + + function switchEngineMode(mode) { + setEngineMode(mode, { projectDir, onTransition: handleModeTransition }); + return getEngineMode(); + } + // ── In-memory session stats (not written to JSONL to keep log entries small) ── const sessionStats = { byTool: {}, // tool → { calls, errors, totalMs } @@ -72,6 +258,12 @@ export async function createServer({ projectDir, httpPort = 0 }) { if (input?.file_path) m.file_path = input.file_path; if (Array.isArray(output?.errors)) m.error_count = output.errors.length; if (Array.isArray(output?.warnings)) m.warning_count = output.warnings.length; + { const checks = []; + for (const d of [...(output?.errors ?? []), ...(output?.warnings ?? [])]) { + if (d.check && !checks.includes(d.check)) checks.push(d.check); + } + if (checks.length) m.checks = checks; + } break; case 'validate_intent': if (Array.isArray(output?.pending_files)) m.file_count = output.pending_files.length; @@ -90,10 +282,95 @@ export async function createServer({ projectDir, httpPort = 0 }) { return m; } + // Mirror a legacy emit() call into the typed session-event bus. Returns + // void; never throws. Unmapped legacy events are silently dropped (the + // bus only persists kinds it knows how to project — extra log/diagnostic + // events still go to the JSONL logger via rawEmit). + function mirrorToBus(event, data, ts) { + if (sessionBus.isClosed) return; + switch (event) { + case 'server_start': + sessionBus.emit('server_start', { + project_dir: data.projectDir ?? projectDir, + version: VERSION, + http_port: data.httpPort ?? null, + started_at: ts, + }, ts); + return; + case 'server_stop': + sessionBus.emit('server_stop', { reason: data.reason ?? 'unknown' }, ts); + return; + case 'pos_cli_found': + sessionBus.emit('pos_cli_resolved', { + found: true, + path: data.path ?? null, + data_dir: data.dataDir ?? null, + }, ts); + return; + case 'pos_cli_error': + sessionBus.emit('pos_cli_resolved', { found: false, error: data.error ?? null }, ts); + return; + case 'lsp_ready': + sessionBus.emit('lsp_event', { phase: 'ready', duration_ms: data.durationMs }, ts); + return; + case 'lsp_warmed_up': + sessionBus.emit('lsp_event', { + phase: 'warmed_up', duration_ms: data.durationMs, index_ready: data.indexReady, + }, ts); + return; + case 'lsp_crash': + sessionBus.emit('lsp_event', { + phase: 'crash', + code: data.code ?? null, + signal: data.signal ?? null, + restart_count: data.restartCount ?? 0, + }, ts); + return; + case 'lsp_init_failed': + sessionBus.emit('lsp_event', { phase: 'init_failed', error: data.error ?? '' }, ts); + return; + case 'lsp_restart_requested': + sessionBus.emit('lsp_event', { phase: 'restart_requested' }, ts); + return; + case 'lsp_restart_failed': + sessionBus.emit('lsp_event', { phase: 'restart_failed', error: data.error ?? '' }, ts); + return; + case 'index_ready': { + const payload = { index: data.index, status: 'ready' }; + if (data.count != null) payload.count = data.count; + if (data.queries != null) payload.queries = data.queries; + if (data.mutations != null) payload.mutations = data.mutations; + sessionBus.emit('index_event', payload, ts); + return; + } + case 'index_failed': + sessionBus.emit('index_event', { index: data.index, status: 'failed', error: data.error ?? '' }, ts); + return; + case 'tool_call': + sessionBus.emit('tool_call', { + tool: data.tool, + duration_ms: data.durationMs ?? 0, + success: data.success !== false, + input: data.input, + output: data.output, + ...(data.error ? { error: data.error } : {}), + }, ts); + return; + // fs_change is emitted directly by the watcher via the bus — no mirror. + // Other legacy events (lsp_request, fs_watcher_start_failed, etc.) are + // diagnostic-only and not part of the projection. + } + } + // Wrap emit: track in-memory stats, add lightweight metadata, strip large payloads. // lsp_request events are very frequent and not useful in the log file. function emit(event, data = {}) { if (event === 'lsp_request') return; // too noisy + const ts = new Date().toISOString(); + // Mirror to the typed bus FIRST (before any payload stripping below). + // Wrapped in try/catch so a bus issue can never break the live request path. + try { mirrorToBus(event, data, ts); } catch (e) { log(`session-event-bus mirror error: ${e.message}`); } + if (event === 'tool_call') { trackStats(data.tool, data.durationMs, data.success, data.output); const meta = extractLogMeta(data.tool, data.input, data.output); @@ -105,14 +382,17 @@ export async function createServer({ projectDir, httpPort = 0 }) { ...meta, }; rawEmit(event, entry); - broadcastSse({ event, ts: new Date().toISOString(), ...entry }); + broadcastSse({ event, ts, ...entry }); return; } rawEmit(event, data); - broadcastSse({ event, ts: new Date().toISOString(), ...data }); + broadcastSse({ event, ts, ...data }); } - rawEmit('server_start', { projectDir, httpPort }); + const serverStartMs = Date.now(); + // emit() mirrors to the session bus; rawEmit would skip the bus and we'd + // miss server_start in the NDJSON log (and replay would never see it). + emit('server_start', { projectDir, httpPort }); log(`Starting pos-supervisor v${VERSION} for ${projectDir}`); // ── Resolve pos-cli paths ───────────────────────────────────────────────── @@ -347,7 +627,7 @@ export async function createServer({ projectDir, httpPort = 0 }) { // validate_code/analyze_project read it and merge with explicit params. // scaffold(write:true) clears it after the files land on disk. const session = { - fileHistory: new Map(), // filePath → { calls, lastErrorCount, consecutiveNonDecreasing } + fileHistory: new Map(), // filePath → { calls, lastErrorCount, consecutiveNonDecreasing, lastChecks } validatedPlan: null, // { planId, pendingFiles: Set, validatedFiles: Set } (legacy — see pending) pending: { files: new Set(), @@ -357,6 +637,11 @@ export async function createServer({ projectDir, httpPort = 0 }) { validatedAt: null, writeDirectly: false, // true after successful scaffold_output validation (trusted track) }, + checkEffectiveness: {}, // check → { fixed, stuck } — transitions between consecutive validate_code calls + scaffoldRuns: [], // [{ ts, model, type, files: string[] }] + enrichHistory: [], // [{ file, check, ts }] — pending enrich_error calls awaiting validate_code + hintEffectiveness: {}, // check → { hinted, fixedAfterHint } — hint-then-fix correlation + pipelineTraces: new Map(), // filePath → trace[] — most recent pipeline trace per file }; const ctx = { @@ -374,8 +659,14 @@ export async function createServer({ projectDir, httpPort = 0 }) { filtersIndex, tagsIndex, session, + sessionBus, + blobStore, + analyticsStore, + cacConfigState, log, emit, + switchEngineMode, + getEngineMode, }; // ── Create MCP server (SDK) for stdio transport ─────────────────────────── @@ -400,6 +691,41 @@ export async function createServer({ projectDir, httpPort = 0 }) { setTimeout(() => shutdown('stdin-closed'), 200); }); + // ── Session persistence (D3 — comparative session view) ─────────────────── + // sessionsDir + sessionId are declared above (Phase A1 event bus) so the + // bus can open its NDJSON writer before the first emit fires. + + function saveSessionSummary() { + try { + mkdirSync(sessionsDir, { recursive: true }); + const stats = sessionStats.byTool; + let totalCalls = 0, totalErrors = 0; + for (const s of Object.values(stats)) { + totalCalls += s.calls || 0; + totalErrors += s.errors || 0; + } + const summary = { + id: sessionId, + startedAt: new Date(serverStartMs).toISOString(), + endedAt: new Date().toISOString(), + projectDir, + version: VERSION, + toolCalls: totalCalls, + toolErrors: totalErrors, + filesValidated: session.fileHistory.size, + checkFrequency: sessionStats.checkFrequency, + checkEffectiveness: session.checkEffectiveness, + hintEffectiveness: session.hintEffectiveness, + scaffoldRuns: session.scaffoldRuns.length, + stats, + }; + writeFileSync(join(sessionsDir, `${sessionId}.json`), JSON.stringify(summary, null, 2)); + log(`Session summary saved: ${sessionId}`); + } catch (e) { + log(`Session save failed: ${e.message}`); + } + } + // ── Start HTTP transport (optional, for REST consumers and tests) ───────── if (httpPort > 0) { const startMs = Date.now(); @@ -426,18 +752,31 @@ export async function createServer({ projectDir, httpPort = 0 }) { calls: h.calls, lastErrorCount: h.lastErrorCount, lastWarningCount: h.lastWarningCount ?? 0, + lastChecks: h.lastChecks || [], + prevChecks: h.prevChecks || [], + consecutiveNonDecreasing: h.consecutiveNonDecreasing ?? 0, })), + checkEffectiveness: session.checkEffectiveness, + scaffoldRuns: session.scaffoldRuns, + hintEffectiveness: session.hintEffectiveness, + pipelineTraces: [...session.pipelineTraces.entries()].map(([path, trace]) => ({ path, trace })), + analytics: analyticsStore ? analyticsStore.stats() : null, + engineMode: getEngineMode(), }; } const dataRoot = join(__dirname, 'data'); - startHttp(registry, { port: httpPort, log, version: VERSION, logPath, getStatus, restartLsp, dataRoot, subscribeToEvents, posCliPath, projectDir }); + startHttp(registry, { port: httpPort, log, version: VERSION, logPath, getStatus, restartLsp, dataRoot, subscribeToEvents, posCliPath, projectDir, sessionsDir, saveSessionSummary, analyticsStore, blobStore, onAnalyticsRebuild: syncDisabledRules, onOverridesChanged: () => { syncRuleOverrides(); syncDisabledRules(); }, onCacConfigChanged: syncCacConfig, switchEngineMode, getEngineMode }); } // ── Graceful shutdown ───────────────────────────────────────────────────── function shutdown(reason) { emit('server_stop', { reason }); log(`Shutting down (${reason})...`); + try { saveSessionSummary(); } catch {} try { fsWatcher?.close(); } catch {} + try { promotedRulesWatcher?.close(); } catch {} + try { analyticsStore?.close(); } catch {} + try { sessionBus.close(); } catch {} // runs final replay-vs-projection invariant + closes NDJSON lsp.stop(); closeLogger(); mcpServer.close().catch(() => {}); diff --git a/src/tools.js b/src/tools.js index 9459244..40acb16 100644 --- a/src/tools.js +++ b/src/tools.js @@ -69,15 +69,42 @@ export function createToolRegistry(ctx, mcpServer = null) { // Wrap handler with timing telemetry + session tracking const timedHandler = async (args) => { + // Dashboard-originated calls (e.g. Live Diagnostic Console) must NOT + // pollute agent-activity surfaces: File Validation Map, Activity log, + // tool stats, bigram sequences, or NDJSON session log. The sentinel is + // stripped before the raw handler runs so tool logic never sees it. + const untracked = args?._source === 'dashboard_live'; + let cleanArgs = args; + if (args && typeof args === 'object' && '_source' in args) { + const { _source, ...rest } = args; + cleanArgs = rest; + } + + if (untracked) { + // `ctx.untracked = true` is read by tool handlers that emit directly + // to sessionBus (currently validate-code.js → `validator_emit`). The + // flag is transient — restore the previous value in a finally so a + // tool handler that happens to trigger another tool mid-flight does + // not lose state. Single-threaded event loop: concurrent calls + // already share ctx, so this is consistent with existing practice. + const prevUntracked = ctx.untracked; + ctx.untracked = true; + try { + return await rawHandler(cleanArgs); + } finally { + ctx.untracked = prevUntracked; + } + } + const start = Date.now(); let success = true; try { - const result = await rawHandler(args); + const result = await rawHandler(cleanArgs); const durationMs = Date.now() - start; - ctx.emit?.('tool_call', { tool: tool.name, durationMs, success, input: args, output: result }); + ctx.emit?.('tool_call', { tool: tool.name, durationMs, success, input: cleanArgs, output: result }); // Session tracking (non-blocking, best-effort) - try { updateSession(ctx.session, tool.name, args, result); } catch (e) { ctx.log?.(`Session tracking error: ${e.message}`); } + try { updateSession(ctx.session, tool.name, cleanArgs, result); } catch (e) { ctx.log?.(`Session tracking error: ${e.message}`); } // Tool avoidance detection: if there's a validated plan with unvalidated files // and the agent is calling tools other than validation, add an advisory note @@ -98,7 +125,7 @@ export function createToolRegistry(ctx, mcpServer = null) { } catch (e) { success = false; const durationMs = Date.now() - start; - ctx.emit?.('tool_call', { tool: tool.name, durationMs, success, input: args, error: e.message }); + ctx.emit?.('tool_call', { tool: tool.name, durationMs, success, input: cleanArgs, error: e.message }); throw e; } }; @@ -182,6 +209,8 @@ function updateSession(session, toolName, args, result) { const fp = args.file_path; const errorCount = result?.errors?.length ?? 0; const warningCount = result?.warnings?.length ?? 0; + const currentChecks = [...(result?.errors ?? []), ...(result?.warnings ?? [])] + .map(d => d.check).filter(Boolean); const prev = session.fileHistory.get(fp); if (prev) { @@ -191,14 +220,31 @@ function updateSession(session, toolName, args, result) { } else { prev.consecutiveNonDecreasing = 0; } + // Track per-check effectiveness (fixed vs stuck between consecutive calls) + if (session.checkEffectiveness && prev.lastChecks?.length) { + const prevSet = new Set(prev.lastChecks); + const curSet = new Set(currentChecks); + for (const check of prevSet) { + if (!session.checkEffectiveness[check]) session.checkEffectiveness[check] = { fixed: 0, stuck: 0 }; + if (curSet.has(check)) { + session.checkEffectiveness[check].stuck++; + } else { + session.checkEffectiveness[check].fixed++; + } + } + } + prev.prevChecks = prev.lastChecks || []; prev.lastErrorCount = errorCount; prev.lastWarningCount = warningCount; + prev.lastChecks = currentChecks; } else { session.fileHistory.set(fp, { calls: 1, lastErrorCount: errorCount, lastWarningCount: warningCount, consecutiveNonDecreasing: 0, + lastChecks: currentChecks, + prevChecks: [], }); } @@ -210,4 +256,55 @@ function updateSession(session, toolName, args, result) { session.validatedPlan.validatedFiles.add(fp); } } + + // Track scaffold runs for quality scoring + if (toolName === 'scaffold' && result?.files && session.scaffoldRuns) { + session.scaffoldRuns.push({ + ts: new Date().toISOString(), + model: args?.model, + type: args?.type, + files: result.files.map(f => f.path || f).filter(Boolean), + written: result.written || [], + }); + } + + // Track enrich_error calls for hint effectiveness scoring (A2) + if (toolName === 'enrich_error' && args?.file_path && args?.check_name) { + session.enrichHistory.push({ + file: args.file_path, + check: args.check_name, + ts: Date.now(), + }); + } + + // Compute hint effectiveness: when validate_code runs, check if previously + // enriched checks for that file got fixed (A2) + if (toolName === 'validate_code' && args?.file_path && session.enrichHistory.length) { + const fp = args.file_path; + const currentCheckSet = new Set( + [...(result?.errors ?? []), ...(result?.warnings ?? [])].map(d => d.check).filter(Boolean) + ); + const consumed = []; + for (let i = 0; i < session.enrichHistory.length; i++) { + const eh = session.enrichHistory[i]; + if (eh.file !== fp) continue; + consumed.push(i); + if (!session.hintEffectiveness[eh.check]) { + session.hintEffectiveness[eh.check] = { hinted: 0, fixedAfterHint: 0 }; + } + session.hintEffectiveness[eh.check].hinted++; + if (!currentCheckSet.has(eh.check)) { + session.hintEffectiveness[eh.check].fixedAfterHint++; + } + } + // Remove consumed entries (reverse order to preserve indices) + for (let i = consumed.length - 1; i >= 0; i--) { + session.enrichHistory.splice(consumed[i], 1); + } + } + + // Store pipeline trace if present (D2 — pipeline inspector) + if (toolName === 'validate_code' && args?.file_path && result?._pipelineTrace) { + session.pipelineTraces.set(args.file_path, result._pipelineTrace); + } } diff --git a/src/tools/analyze-project.js b/src/tools/analyze-project.js index e93bcb3..467462c 100644 --- a/src/tools/analyze-project.js +++ b/src/tools/analyze-project.js @@ -3,6 +3,7 @@ import { readdir, readFile } from 'node:fs/promises'; import { join } from 'node:path'; import { createCheckRunner } from '../core/check-runner.js'; import { validateSchema } from '../core/schema-validator.js'; +import { validateTranslationYaml } from '../core/translation-validator.js'; import { toUri, sanitizePath } from '../core/utils.js'; import { getProjectMap } from './project-map.js'; import { ToolError } from '../core/tool-error.js'; @@ -11,13 +12,14 @@ import { buildPendingPartialNames, buildPendingPageKeys, } from '../core/diagnostic-pipeline.js'; -import { buildDependencyGraph, detectDeadCode } from '../core/dependency-graph.js'; +import { buildDependencyGraph, detectOrphanedFiles } from '../core/dependency-graph.js'; import { findOrphanPartials } from '../core/orphan-detector.js'; import { resolveRenderName } from '../core/project-scanner.js'; +import { buildFactGraph } from '../core/project-fact-graph.js'; export const analyzeProjectTool = { name: 'analyze_project', - description: 'Cross-file project health overview. Returns per-file error/warning counts, dependency graph, broken references, dead code, and schema issues. Use validate_code on individual files for full diagnostics, fix proposals, and context-aware analysis.', + description: 'Cross-file project health overview. Returns per-file error/warning counts, dependency graph, broken references, orphaned files, and schema issues. Use validate_code on individual files for full diagnostics, fix proposals, and context-aware analysis.', inputSchema: { files: z.array(z.string()).optional().describe('List of file paths (relative to project root) to analyze. Omit to analyze all project files.'), min_severity: z.enum(['error', 'warning', 'info']).optional().describe('Minimum severity to include in file counts. "error" = only list files with errors, "warning" = errors + warnings (default), "info" = everything.'), @@ -36,17 +38,18 @@ export const analyzeProjectTool = { const SEV_RANK = { error: 3, warning: 2, info: 1 }; const minRank = SEV_RANK[min_severity] ?? 2; - // If no files specified, discover all .liquid and .graphql files in app/ + // Fetch the project map once up front — used for file listing, dep graph, + // and integrity checks. Cached by getProjectMap so this is near-free. + const projectMap = await getProjectMap(ctx.directory); + const factGraph = buildFactGraph(projectMap); + + // If no files specified, use the fact graph's indexed file list instead of + // re-walking app/ (eliminates the parallel-walk class of bugs). + // Also include translation files explicitly so they're part of the primary analysis. if (!files || !Array.isArray(files) || files.length === 0) { - const appDir = join(ctx.directory, 'app'); - try { - const entries = await readdir(appDir, { recursive: true }); - files = entries - .filter(e => e.endsWith('.liquid') || e.endsWith('.graphql')) - .map(e => join('app', e)); - } catch { - throw new ToolError('No files specified and could not scan app/ directory', { status: 404 }); - } + const checkableFiles = factGraph.allCheckableFiles(); + const translationFiles = getTranslationFilePaths(projectMap); + files = [...checkableFiles, ...translationFiles]; if (files.length === 0) { throw new ToolError('No .liquid or .graphql files found in app/', { status: 404 }); } @@ -117,6 +120,32 @@ export const analyzeProjectTool = { } } + // Catch diagnostics from pos-cli check that target files outside + // the checkable set (e.g. MatchingTranslations in .yml files). + const fileSet = new Set(files); + const unattributed = new Map(); + for (const d of [...allResults.errors, ...allResults.warnings, ...allResults.infos]) { + if (!d._filePath) continue; + const rel = d._filePath.startsWith(ctx.directory) + ? d._filePath.slice(ctx.directory.length + 1) + : d._filePath; + if (fileSet.has(rel)) continue; + if (!unattributed.has(rel)) unattributed.set(rel, { errors: 0, warnings: 0, infos: 0 }); + const counts = unattributed.get(rel); + counts[d.severity === 'error' ? 'errors' : d.severity === 'warning' ? 'warnings' : 'infos']++; + } + for (const [path, counts] of unattributed) { + const hasRelevant = + counts.errors > 0 || + (minRank <= 2 && counts.warnings > 0) || + (minRank <= 1 && counts.infos > 0); + if (hasRelevant) { + const entry = { path, errors: counts.errors, warnings: counts.warnings }; + if (minRank <= 1) entry.infos = counts.infos; + fileResults.push(entry); + } + } + // Schema validation — validate all .yml files in app/schema/ let schemasScanned = 0; try { @@ -139,6 +168,30 @@ export const analyzeProjectTool = { } } catch { /* schema directory not found — skip */ } + // Translation validation — catch structural invariant violations (e.g. missing + // top-level locale key, stray non-locale top-level keys) that pos-cli check + // does not report as errors on the .yml file itself. Without this step, a + // broken translation file (e.g. `enff:` instead of `en:`) has 0 pos-cli errors + // and never enters fix_order, even though it is the root cause of + // TranslationKeyExists errors on every liquid file that uses translation keys. + for (const tranPath of getTranslationFilePaths(projectMap)) { + try { + const content = await readFile(join(ctx.directory, tranPath), 'utf8'); + const transResult = validateTranslationYaml(content, tranPath); + const errorCount = transResult.errors.length; + const warningCount = transResult.warnings.length; + const hasRelevant = errorCount > 0 || (minRank <= 2 && warningCount > 0); + if (!hasRelevant) continue; + const existing = fileResults.find(f => f.path === tranPath); + if (existing) { + existing.errors += errorCount; + existing.warnings += warningCount; + } else { + fileResults.push({ path: tranPath, errors: errorCount, warnings: warningCount }); + } + } catch { /* translation file read/parse failure — skip */ } + } + // Build dependency graph. // // The LSP's appGraph/* methods return empty arrays for files it has not @@ -153,6 +206,7 @@ export const analyzeProjectTool = { const lspOverlay = {}; if (ctx.lsp?.initialized) { for (const filePath of files) { + if (filePath.endsWith('.yml') || filePath.endsWith('.yaml')) continue; const absPath = absPaths[filePath]; const uri = toUri(absPath); try { @@ -171,22 +225,18 @@ export const analyzeProjectTool = { } } - // Fetch the project map once — both the dep graph and the integrity - // checks consume it. - const projectMap = await getProjectMap(ctx.directory); - // projectMap-derived graph merged with LSP overlay. This is the // authoritative graph surfaced to the agent. const dependencyGraph = buildDependencyGraph(projectMap, lspOverlay); - // Cross-file integrity checks + dead code detection. + // Cross-file integrity checks + orphaned file detection. let integrity = []; - let dead_code = []; + let orphaned_files = []; try { integrity = performIntegrityChecks(projectMap); - dead_code = detectDeadCode(dependencyGraph, projectMap); + orphaned_files = detectOrphanedFiles(dependencyGraph, projectMap); } catch { - // Integrity checks and dead code detection are best-effort. + // Integrity checks and orphaned file detection are best-effort. } // Apply severity filter to integrity issues @@ -194,18 +244,44 @@ export const analyzeProjectTool = { integrity = integrity.filter(i => (SEV_RANK[i.severity] ?? 1) >= minRank); } + // Inject translation dependency edges into dependencyGraph so buildFixOrder + // can place the translation file before the liquid files it breaks. + // filesAffectedByTranslationFile relies on translation_keys in projectMap + // which the scanner doesn't populate — using allResults.errors directly + // gives us the ground truth: every file with a TranslationKeyExists error + // implicitly depends on the broken translation file. + const tranPrefix = ctx.directory.endsWith('/') ? ctx.directory : ctx.directory + '/'; + for (const tranPath of getTranslationFilePaths(projectMap)) { + if (!fileResults.some(f => f.path === tranPath && f.errors > 0)) continue; + for (const d of allResults.errors) { + if (d.check !== 'TranslationKeyExists') continue; + const rel = d._filePath?.startsWith(tranPrefix) + ? d._filePath.slice(tranPrefix.length) + : d._filePath; + if (!rel || rel === tranPath) continue; + if (!dependencyGraph[rel]) dependencyGraph[rel] = { depends_on: [], referenced_by: [] }; + if (!dependencyGraph[rel].depends_on.includes(tranPath)) { + dependencyGraph[rel].depends_on.push(tranPath); + } + if (!dependencyGraph[tranPath]) dependencyGraph[tranPath] = { depends_on: [], referenced_by: [] }; + if (!dependencyGraph[tranPath].referenced_by.includes(rel)) { + dependencyGraph[tranPath].referenced_by.push(rel); + } + } + } + const lintErrors = fileResults.reduce((s, f) => s + f.errors, 0); const lintWarnings = fileResults.reduce((s, f) => s + f.warnings, 0); const integrityErrors = integrity.filter(i => i.severity === 'error').length; const integrityWarnings = integrity.filter(i => i.severity === 'warning').length; - const fix_order = buildFixOrder(fileResults, dependencyGraph, ctx.directory); + const fix_order = buildFixOrder(fileResults, dependencyGraph, ctx.directory, projectMap); const totalErrors = lintErrors + integrityErrors; const totalWarnings = lintWarnings + integrityWarnings; // ── blocking_files: files with errors that must be fixed ──────────── - const blockingFiles = computeBlockingFiles(fileResults, integrity); + const blockingFiles = computeBlockingFiles(fileResults, integrity, allResults, ctx.directory, projectMap); // ── diff_from_last_run: compare against previous analysis ────────── const prefix = ctx.directory.endsWith('/') ? ctx.directory : ctx.directory + '/'; @@ -225,13 +301,13 @@ export const analyzeProjectTool = { } return { - files_scanned: filesScanned + schemasScanned, + files_scanned: filesScanned + schemasScanned + unattributed.size, files: fileResults, fix_order, blocking_files: blockingFiles, diff_from_last_run: diff, dependency_graph: dependencyGraph, - dead_code, + orphaned_files, integrity, lint_errors: lintErrors, lint_warnings: lintWarnings, @@ -258,12 +334,16 @@ export const analyzeProjectTool = { * If A renders/calls B and both have errors, B should be fixed first — * fixing B may eliminate cascade errors in A. * + * Translation files are handled specially: they have implicit dependents + * (all files that use translation keys). These are computed at analyze time. + * * @param {{ path: string, errors: number, warnings: number }[]} fileResults * @param {Record} dependencyGraph * @param {string} projectDir - absolute project root + * @param {object} projectMap - project indexing result (for translation file handling) * @returns {{ path: string, errors: number, warnings: number, reason: string, dependents_with_errors: number }[]} */ -export function buildFixOrder(fileResults, dependencyGraph, projectDir) { +export function buildFixOrder(fileResults, dependencyGraph, projectDir, projectMap) { if (fileResults.length === 0) return []; const errorPaths = new Set(fileResults.map(f => f.path)); @@ -295,6 +375,20 @@ export function buildFixOrder(fileResults, dependencyGraph, projectDir) { } } + // Handle translation file implicit dependents: files that use translation keys + // depend on translation files, so if a translation file has errors, all its + // dependents should be fixed after it. + for (const tranFile of fileResults.filter(f => f.path.startsWith('app/translations/'))) { + const affectedFiles = filesAffectedByTranslationFile(tranFile.path, projectMap); + for (const affectedPath of affectedFiles) { + if (errorPaths.has(affectedPath)) { + // affectedPath depends on tranFile + deps[affectedPath].add(tranFile.path); + dependents[tranFile.path].add(affectedPath); + } + } + } + // Kahn's algorithm — nodes with in-degree 0 (no unresolved deps) go first. const inDegree = {}; for (const f of fileResults) inDegree[f.path] = deps[f.path].size; @@ -416,12 +510,18 @@ function performIntegrityChecks(projectMap) { } // 3. Broken function calls (from pages, partials, commands, queries) - // In platformOS, {% function result = 'queries/X' %} resolves to app/lib/queries/X.liquid - // The lib/ prefix is implicit in function calls. + // + // platformOS resolves `function` paths relative to the partial search + // paths declared by `@platformos/platformos-common`: + // FILE_TYPE_DIRS[Partial] = ['views/partials', 'lib'] + // joined under `app/`. So `'commands/X'` resolves to `app/lib/commands/X.liquid`, + // and `'lib/commands/X'` resolves to `app/lib/lib/commands/X.liquid` + // (which never exists). A literal `lib/` prefix is *invalid*, not optional. + // Reporting the resolution verbatim — without stripping `lib/` — surfaces + // the bug to the agent through the error message itself. const checkFunctionCalls = (sourcePath, functionCalls) => { for (const fc of functionCalls ?? []) { if (isModuleRef(fc.path)) continue; - // function call path → disk path: app/lib/{path}.liquid const fullPath = `app/lib/${fc.path}.liquid`; if (fc.path.includes('commands/') && !allCommands.has(fullPath)) { issues.push({ @@ -437,16 +537,29 @@ function performIntegrityChecks(projectMap) { } }; - for (const [slug, page] of Object.entries(projectMap.pages ?? {})) { + for (const [, page] of Object.entries(projectMap.pages ?? {})) { checkFunctionCalls(page.path, page.function_calls); } - // Also check function calls from partials, commands, and queries - for (const [name, partial] of Object.entries(projectMap.partials ?? {})) { + for (const [, partial] of Object.entries(projectMap.partials ?? {})) { checkFunctionCalls(partial.path, partial.function_calls); } + // Commands invoke their own build/check phases via {% function %}; queries + // and layouts also carry function_calls in the project map. Without these, + // a wrong call inside a command (the most common form: a multi-phase + // command calling its sibling phase with `lib/commands/...`) slips through + // unchecked. + for (const [path, cmd] of Object.entries(projectMap.commands ?? {})) { + checkFunctionCalls(path, cmd.function_calls); + } + for (const [path, query] of Object.entries(projectMap.queries ?? {})) { + checkFunctionCalls(path, query.function_calls); + } + for (const [, layout] of Object.entries(projectMap.layouts ?? {})) { + checkFunctionCalls(layout.path, layout.function_calls); + } // 4. Orphan partials (never rendered by anything) — shared predicate so - // validate_intent P5 and dependency-graph dead-code detection use the + // validate_intent P5 and dependency-graph orphaned-file detection use the // same rule. See src/core/orphan-detector.js. for (const { name, path } of findOrphanPartials(projectMap)) { issues.push({ @@ -472,13 +585,51 @@ function matchesFile(diagnostic, absPath, relPath) { /** * Files with at least one error (lint or integrity) that block a clean project. * Sorted by total error count descending — worst offenders first. + * @param {Array} fileResults - per-file { path, errors, warnings } + * @param {Array} integrity - integrity issues with { severity, source, type } + * @param {{ errors: Array }} [allResults] - full diagnostic results for check name extraction + * @param {string} [projectDir] - absolute project root, needed for translation file path normalisation + * @param {object} [projectMap] - project indexing result, needed to discover translation files */ -export function computeBlockingFiles(fileResults, integrity) { +export function computeBlockingFiles(fileResults, integrity, allResults, projectDir, projectMap) { const blockMap = new Map(); for (const f of fileResults) { if (f.errors > 0) { - blockMap.set(f.path, { path: f.path, lint_errors: f.errors, integrity_errors: 0 }); + blockMap.set(f.path, { path: f.path, lint_errors: f.errors, integrity_errors: 0, checks: new Set() }); + } + } + + if (allResults?.errors) { + for (const d of allResults.errors) { + const rel = d._filePath; + for (const [key, entry] of blockMap) { + if (rel && (rel === key || rel.endsWith('/' + key) || rel.endsWith(key))) { + if (d.check) entry.checks.add(d.check); + } + } + } + } + + // Explicitly ensure translation file errors reach blocking_files even when the + // caller provided an explicit files list that excluded translation files. + // (When files come from the default discovery path, translation files are already + // in fileResults via Change 1a — this handles the explicit-files-list case.) + if (projectDir && projectMap && allResults?.errors) { + const prefix = projectDir.endsWith('/') ? projectDir : projectDir + '/'; + for (const tranPath of getTranslationFilePaths(projectMap)) { + if (blockMap.has(tranPath)) continue; + const errors = allResults.errors.filter(d => { + if (!d._filePath) return false; + const rel = d._filePath.startsWith(prefix) + ? d._filePath.slice(prefix.length) + : d._filePath; + return rel === tranPath; + }); + if (errors.length > 0) { + const checks = new Set(errors.map(e => e.check).filter(Boolean)); + blockMap.set(tranPath, { path: tranPath, lint_errors: errors.length, integrity_errors: 0, checks }); + } } } @@ -487,13 +638,16 @@ export function computeBlockingFiles(fileResults, integrity) { const existing = blockMap.get(issue.source); if (existing) { existing.integrity_errors++; + if (issue.type) existing.checks.add(issue.type); } else { - blockMap.set(issue.source, { path: issue.source, lint_errors: 0, integrity_errors: 1 }); + const checks = new Set(); + if (issue.type) checks.add(issue.type); + blockMap.set(issue.source, { path: issue.source, lint_errors: 0, integrity_errors: 1, checks }); } } return [...blockMap.values()] - .map(b => ({ ...b, total: b.lint_errors + b.integrity_errors })) + .map(b => ({ ...b, total: b.lint_errors + b.integrity_errors, checks: [...b.checks] })) .sort((a, b) => b.total - a.total); } @@ -552,3 +706,47 @@ export function computeDiffFromLastRun(session, currentSnapshot, totalErrors, to warning_delta: totalWarnings - (prev.total_warnings ?? 0), }; } + +/** + * Get the list of translation file paths from the project map. + * @param {object} projectMap - project indexing result + * @returns {string[]} relative paths like 'app/translations/en.yml' + */ +export function getTranslationFilePaths(projectMap) { + return Object.keys(projectMap.translations || {}).map( + locale => `app/translations/${locale}.yml` + ); +} + +/** + * Find all .liquid files that would be affected if a translation file has errors. + * Translation file errors (e.g., missing locale key, mismatched keys across locales) + * cause TranslationKeyExists failures on any file that references keys from that locale. + * + * @param {string} translationFilePath - e.g., 'app/translations/en.yml' + * @param {object} projectMap - project indexing result + * @returns {Set} relative paths of files that depend on this translation file + */ +export function filesAffectedByTranslationFile(translationFilePath, projectMap) { + const affected = new Set(); + const locale = translationFilePath.match(/\/(\w+)\.yml$/)?.[1]; + if (!locale) return affected; + + // Collect all files that use translation keys — these depend on the translation file + const allFiles = [ + ...Object.values(projectMap.pages || {}), + ...Object.values(projectMap.partials || {}), + ...Object.values(projectMap.commands || {}), + ...Object.values(projectMap.queries || {}), + ]; + + for (const file of allFiles) { + // If file has any translation key references, it depends on the translation file + if (file.translation_keys && file.translation_keys.length > 0) { + affected.add(file.path); + } + } + + return affected; +} + diff --git a/src/tools/module-info.js b/src/tools/module-info.js index fbb13b5..7fccf36 100644 --- a/src/tools/module-info.js +++ b/src/tools/module-info.js @@ -92,6 +92,8 @@ export const moduleInfoTool = { name, version: scan.version, dependencies: scan.dependencies, + ...(scan.manifest_source ? { manifest_source: scan.manifest_source } : {}), + ...(scan.manifest_warnings ? { manifest_warnings: scan.manifest_warnings } : {}), section: 'api', // Live scan is the source of truth — every entry includes call_syntax, // required_params, optional_params, and returns when the doc block @@ -114,6 +116,8 @@ export const moduleInfoTool = { name, version: scan.version, dependencies: scan.dependencies, + ...(scan.manifest_source ? { manifest_source: scan.manifest_source } : {}), + ...(scan.manifest_warnings ? { manifest_warnings: scan.manifest_warnings } : {}), section, reference: refDoc || `No ${section} documentation available for module '${name}'.`, // Include API surface for quick context @@ -169,6 +173,8 @@ async function buildOverview(name, scan) { display_name: scan.display_name, version: scan.version, dependencies: scan.dependencies, + ...(scan.manifest_source ? { manifest_source: scan.manifest_source } : {}), + ...(scan.manifest_warnings ? { manifest_warnings: scan.manifest_warnings } : {}), installed: true, // Required setup steps — always surfaced when docs exist so agents don't miss prerequisites diff --git a/src/tools/project-map.js b/src/tools/project-map.js index 03e87f3..2035093 100644 --- a/src/tools/project-map.js +++ b/src/tools/project-map.js @@ -16,6 +16,7 @@ export const projectMapTool = { Returns a structured JSON index of the platformOS project: schemas with property details, GraphQL operations with args, commands, queries, pages, partials with reverse-index, translations, and per-resource CRUD completeness. +You MUST run project_map with force_refresh: true after creating new commands, otherwise the LSP may has stale/cached state. MANDATE (NON-NEGOTIABLE): Call this tool ONCE at the very start of every session before using scaffold, validate_intent, diff --git a/src/tools/server-status.js b/src/tools/server-status.js index 4463358..5cb8e7b 100644 --- a/src/tools/server-status.js +++ b/src/tools/server-status.js @@ -1,4 +1,6 @@ import { VALID_DOMAINS } from '../core/domain-detector.js'; +import { ruleScores } from '../core/case-base.js'; +import { getEngineMode } from '../core/engine-mode.js'; export const serverStatusTool = { name: 'server_status', @@ -8,9 +10,22 @@ export const serverStatusTool = { createHandler(ctx) { return async () => { const pending = ctx.session?.pending ?? null; + let disabledRules = []; + if (ctx.analyticsStore) { + try { + // Engine state snapshot: bypass any operator reporting baseline. + // The disabled-rules list must reflect the case-base's full-history + // verdict — same data the runtime uses for syncDisabledRules. + // `since: null` is the explicit bypass; see case-base.ruleScores. + disabledRules = ruleScores(ctx.analyticsStore, { minEmitted: 10, since: null }) + .filter(s => s.disabled) + .map(s => ({ rule_id: s.rule_id, effectiveness: s.effectiveness, total_outcomes: s.total_outcomes })); + } catch { /* non-fatal */ } + } return { server: 'pos-supervisor', version: ctx.version, + engine_mode: getEngineMode(), project_dir: ctx.directory, pos_cli: { found: ctx.posCliFound ?? false, @@ -43,6 +58,8 @@ export const serverStatusTool = { }, tip: 'MUST call load_development_guide and domain_guide(domain) for every domain you touch BEFORE writing code. Covers gotchas, patterns, and auth rules not in training data. Skipping these is the #1 cause of broken platformOS code.', + disabled_rules: disabledRules, + session_pending: pending ? { files: [...pending.files], translations: [...pending.translations], diff --git a/src/tools/validate-code.js b/src/tools/validate-code.js index 34a21af..f97961c 100644 --- a/src/tools/validate-code.js +++ b/src/tools/validate-code.js @@ -3,19 +3,27 @@ import { existsSync, readFileSync } from 'node:fs'; import { parseLiquidFile, extractAllFromAST } from '../core/liquid-parser.js'; import { checkContent } from '../core/check-runner.js'; import { normalizeLspDiagnostics } from '../core/lsp-client.js'; -import { enrichAll } from '../core/error-enricher.js'; +import { enrichAll, bridgeRulesOntoUnattributed } from '../core/error-enricher.js'; import { generateFixes, clusterDiagnostics, generateScorecard } from '../core/fix-generator.js'; import { getDomainFromPath, getDomainHeader } from '../core/domain-detector.js'; import { getTriggeredGotchas, getContentTriggers } from '../core/knowledge-loader.js'; import { generateStructuralWarnings } from '../core/structural-warnings.js'; import { validateSchema } from '../core/schema-validator.js'; +import { validateTranslationYaml } from '../core/translation-validator.js'; import { checkSchemaProperties } from '../core/schema-property-checker.js'; -import { runDiagnosticPipeline } from '../core/diagnostic-pipeline.js'; +import { runDiagnosticPipeline, stampDefaultsOn, suppressUpstreamFrontmatterDup } from '../core/diagnostic-pipeline.js'; +import { isCheckForceDisabled } from '../core/rules/engine.js'; +import { applyCac } from '../core/cac-predictor.js'; import { partitionCallersByPending } from '../core/pending-callers.js'; import { toUri, sanitizePath } from '../core/utils.js'; +import { fingerprint, templateFingerprint, messageTemplate, extractParams } from '../core/diagnostic-record.js'; import { getProjectMap } from './project-map.js'; +import { buildFactGraph } from '../core/project-fact-graph.js'; +import { loadAllRules } from '../core/rules/index.js'; import { LSP_DIAGNOSTICS_TIMEOUT_MS, CONSECUTIVE_ERROR_THRESHOLD } from '../core/constants.js'; +loadAllRules(); + /** * Warnings that MUST block a write even when result.status is 'warning'. * @@ -33,7 +41,7 @@ const BLOCKING_WARNINGS = new Set([ 'pos-supervisor:RemovedRender', // removing render breaks user-visible behavior 'pos-supervisor:RemovedGraphQL', // removing graphql call drops data fetch 'pos-supervisor:RemovedParam', // removing @param breaks callers - 'OrphanedPartial', // not reachable — shipping means dead code + 'OrphanedPartial', // not reachable — shipping means orphaned file ]); export const validateCodeTool = { @@ -140,6 +148,7 @@ explicitly only if you are validating a file that is NOT part of the most recent const isLiquid = file_path.endsWith('.liquid'); const isGraphql = file_path.endsWith('.graphql'); const isSchema = file_path.endsWith('.yml') && /(?:^|\/)app\/schema\//.test(file_path); + const isTranslationYaml = /\.ya?ml$/.test(file_path) && /(?:^|\/)app\/translations\//.test(file_path); const result = { errors: [], @@ -219,6 +228,9 @@ explicitly only if you are validating a file that is NOT part of the most recent await ctx.awaitLsp(); } + const projectMap = await getProjectMap(ctx.directory); + const factGraph = buildFactGraph(projectMap); + const enrichCtx = { uri, lsp: ctx.lsp, @@ -226,7 +238,11 @@ explicitly only if you are validating a file that is NOT part of the most recent objectsIndex: ctx.objectsIndex, tagsIndex: ctx.tagsIndex, schemaIndex: ctx.schemaIndex, + analyticsStore: ctx.analyticsStore, content, + factGraph, + filePath: file_path, + projectDir: ctx.directory, }; // Enrich all diagnostics in both quick and full modes. @@ -297,6 +313,21 @@ explicitly only if you are validating a file that is NOT part of the most recent } } + // 2b1. Translation YAML structural validation — catches the missing + // top-level locale key case (`app:` at root instead of `en: → app:`). + // The LSP won't flag this because the YAML parses fine, but every + // `{{ 'key' | t }}` lookup will silently return the raw key. Runs before + // the GraphQL/structural branches so the error lands on the file itself. + if (isTranslationYaml) { + try { + const transResult = validateTranslationYaml(content, file_path); + result.errors.push(...transResult.errors); + result.warnings.push(...transResult.warnings); + } catch (e) { + result.infos.push({ check: 'translation-validator', severity: 'info', message: `Translation validation failed: ${e.message}` }); + } + } + // 2b2. Schema property cross-check (GraphQL files only) if (isGraphql) { try { @@ -342,6 +373,15 @@ explicitly only if you are validating a file that is NOT part of the most recent } } + // 2c1. Drop upstream `ValidFrontmatter` rows that share a line with our + // richer `pos-supervisor:InvalidLayout` / `pos-supervisor:InvalidFrontMatter` + // diagnostics. pos-cli 6.0.7 ships `ValidFrontmatter` independently — + // without this dedup the agent sees two warnings for the same root + // cause (missing layout file, unknown frontmatter key). Upstream rows + // covering cases our checks don't handle (deprecated `layout_name`, + // missing required fields per file type) survive untouched. + suppressUpstreamFrontmatterDup(result); + // 2d. Diff-aware comparison — detect removed functionality on update (full mode) if (isLiquid && fileExists && result.structural && mode === 'full') { try { @@ -497,15 +537,71 @@ explicitly only if you are validating a file that is NOT part of the most recent ctx.directory, ); - result.proposed_fixes = proposedFixes; + // Precedence rule (2026-04-25): the rule engine is the source of + // truth for diagnostic-specific advice; the heuristic generator is + // the source of truth for actionable text_edits derived from + // AST/line position. When BOTH produce output for the same + // diagnostic, we merge them per-channel: + // + // - Heuristic `text_edit` → ALWAYS keep (actionable; complements + // any rule guidance). + // - Heuristic `guidance` → DROP if the rule already emitted any + // fix for this diagnostic. Otherwise + // keep (rule was silent → heuristic is + // the only signal). + // - Rule fixes → ALWAYS keep (priority-ordered; first + // match wins inside the rule engine). + // + // Without this gate, the agent saw competing guidance for the same + // root cause (rule's stale Levenshtein vs heuristic's specific-case + // detection), which actively misled fixes. See the 2026-04-25 + // TranslationKeyExists `[index]` regression report. + const rulesByDiag = new Map(); + for (const d of allDiagnostics) { + if (d.fixes?.length > 0) { + rulesByDiag.set(d, d.fixes); + } + } + // Build a Set of heuristic fix references owned by diagnostics that + // ALSO have rule fixes — those are the ones we'll filter to keep + // only text_edit (drop guidance). + const heuristicByDiagIdx = new Map(diagnosticFixes); // copy + const dropHeuristicGuidance = new Set(); + for (const [diagIdx, hFix] of heuristicByDiagIdx) { + const d = allDiagnostics[diagIdx]; + if (rulesByDiag.has(d) && hFix?.type === 'guidance') { + dropHeuristicGuidance.add(hFix); + } + } + result.proposed_fixes = proposedFixes.filter(f => !dropHeuristicGuidance.has(f)); + + // Merge rule-generated fixes into proposed_fixes + for (const d of allDiagnostics) { + if (d.fixes?.length > 0) { + for (const f of d.fixes) { + result.proposed_fixes.push({ + ...f, + source: 'rule', + rule_id: d.rule_id ?? null, + check: d.check ?? null, + }); + } + } + } - // Attach per-diagnostic fix field - for (const [diagIdx, fix] of diagnosticFixes) { + // Attach per-diagnostic fix field — but if the rule won precedence + // and the heuristic was guidance-only, attach the rule's first fix + // instead so error.fix matches what proposed_fixes carries. + for (const [diagIdx, fix] of heuristicByDiagIdx) { const d = allDiagnostics[diagIdx]; + const ruleFixes = rulesByDiag.get(d); + const useFix = (ruleFixes && fix?.type === 'guidance') + ? { ...ruleFixes[0], source: 'rule', rule_id: d.rule_id ?? null } + : fix; if (d._origType === 'error') { - result.errors[d._origIdx].fix = fix; + result.errors[d._origIdx].fix = useFix; } else { - result.warnings[d._origIdx].fix = fix; + result.warnings[d._origIdx].fix = useFix; } } } catch (e) { @@ -640,12 +736,147 @@ explicitly only if you are validating a file that is NOT part of the most recent if (d.endLine != null) d.endLine += 1; } + // 12a. Run rule-engine rules against diagnostics that didn't pass through + // enrichAll (structural warnings, schema / translation / GraphQL + // validators, diff-aware RemovedRender/AddedParam, new-partial + // caller check). Rule modules for structural checks (like + // `pos-supervisor:NonGetRenderingPage`) register against the check + // name but only fire inside error-enricher.enrichAll, which ran + // before these diagnostics were pushed. This bridge lets their + // rules fire, attaching the rule_id + hint_md that would otherwise + // be lost — see the 2026-04-24 DEMO report finding. + try { + const projectMapForBridge = await getProjectMap(ctx.directory); + const factGraphForBridge = buildFactGraph(projectMapForBridge); + bridgeRulesOntoUnattributed(result, { + filePath: file_path, + content, + factGraph: factGraphForBridge, + filtersIndex: ctx.filtersIndex, + objectsIndex: ctx.objectsIndex, + tagsIndex: ctx.tagsIndex, + schemaIndex: ctx.schemaIndex, + analyticsStore: ctx.analyticsStore, + projectDir: ctx.directory, + }); + } catch { /* bridge is best-effort — fall through to default stamping */ } + + // 12b. Re-stamp default confidence + rule_id across the final diagnostic + // set. runDiagnosticPipeline already ran this as its last step, but + // several sources push into result.errors / result.warnings AFTER + // the pipeline finishes (structural warnings, schema validation, + // translation YAML check, diff-aware RemovedRender/AddedParam, the + // new-partial caller check). Without this second pass those late + // additions land in the analytics store with confidence = null and + // no rule_id, breaking the calibration chart and rule-performance + // attribution. Idempotent — the helper only fills when a field is + // missing. See the 2026-04-23 DEMO report finding. + stampDefaultsOn(result); + + // 12b. Apply force-disable overrides at the check-name level. + // + // runRules() already honors force-disable for rule_ids, but many + // diagnostics originate outside the rule engine — structural + // warnings (pos-supervisor:*), LSP checks without a rule module. + // An operator who force-disables a check like "pos-supervisor:HtmlInPage" + // expects the diagnostic to stop appearing entirely, not just the + // (nonexistent) rule to stop firing. Filter here so the override + // semantics match operator intent. Also covers rule_id-level + // disables as a belt-and-braces second gate (a rule that fires + // despite _forceDisabled containing its id gets dropped here too). + const dropForceDisabled = (d) => + !(isCheckForceDisabled(d.check) || isCheckForceDisabled(d.rule_id)); + result.errors = result.errors.filter(dropForceDisabled); + result.warnings = result.warnings.filter(dropForceDisabled); + result.infos = result.infos.filter(dropForceDisabled); + + // 12c. CAC predictor — opt-in 4th gating axis (Cohen's Agentic Conjecture). + // + // Predicts P(adopted | rule_id, file_domain) from the analytics + // store and either suppresses or downgrades emits whose predicted + // adoption falls below the configured threshold. Disabled by + // default; enabled per project via the dashboard. When disabled, + // this step is a no-op (`applyCac` returns immediately on + // !config.enabled). When enabled in `shadow` mode, decisions are + // recorded for analysis but no diagnostics are mutated. Skipped + // for live-console calls (ctx.untracked) so experimental runs + // don't fight the gate. + // + // Wrapped in try/catch — predictor failure must NEVER break + // validate_code. The whole layer is decoupled in src/core/cac-*. + const cacConfig = ctx.cacConfigState?.current; + if (cacConfig?.enabled && !ctx.untracked) { + try { + applyCac(result, { + config: cacConfig, + analyticsStore: ctx.analyticsStore, + filePath: file_path, + sessionBus: ctx.sessionBus, + log: ctx.log, + }); + } catch (e) { + ctx.log?.(`cac-predictor: applyCac threw (${e?.message ?? e}); diagnostics passed through`); + } + } + // 12. Strip null hint fields — diagnostics without hints should omit the field // entirely rather than returning hint: null which looks like a bug in the output. for (const d of [...result.errors, ...result.warnings, ...result.infos]) { if (d.hint === null || d.hint === undefined) delete d.hint; } + // 13. Emit validator_emit events — one per diagnostic shown to the agent. + // Best-effort: failures never propagate into the tool response. + // Skipped when ctx.untracked is set (dashboard live-console calls) so + // experimental validations don't pollute the analytics store. + if (ctx.sessionBus && !ctx.untracked) { + try { + const contentHash = ctx.blobStore ? ctx.blobStore.put(content) : null; + for (const d of [...result.errors, ...result.warnings]) { + const tmpl = messageTemplate(d.message || ''); + const fp = fingerprint(d.check, file_path, tmpl); + const tFp = templateFingerprint(d.check, tmpl); + const hintHash = d.hint && ctx.blobStore ? ctx.blobStore.put(d.hint) : null; + // Union both fix channels so analytics sees every proposal the agent + // saw: rule engine → d.fixes (plural), heuristic fix-generator → d.fix + // (singular). Both use the same { type, range, new_text, ... } shape. + // Each fix carries its own rule_id so Rule Performance can attribute + // adoption to a specific rule-engine or heuristic-generator branch + // (I1). The rule-engine path stamps rule_id on the rule's HintResult; + // the heuristic path stamps `heuristic:.` centrally + // inside generateFixes. + const diagFixes = [ + ...(Array.isArray(d.fixes) ? d.fixes : []), + ...(d.fix ? [d.fix] : []), + ]; + const fixes = diagFixes.map(f => ({ + range: f.range ?? null, + new_text_hash: ctx.blobStore && f.new_text ? ctx.blobStore.put(f.new_text) : null, + kind: f.type || 'unknown', + // Rule-engine rules attach rule_id to the HintResult (and therefore + // to d.rule_id), not to each individual fix object. Heuristic + // fixes carry their own rule_id from fix-generator's central stamp. + // Falling back to d.rule_id lets rule-engine fixes inherit the + // matching rule's id so fixRulePerformance can attribute them. + rule_id: f.rule_id ?? d.rule_id ?? null, + })); + const diagParams = extractParams(d.check, d.message || ''); + ctx.sessionBus.emit('validator_emit', { + fp, + template_fp: tFp, + file: file_path, + check: d.check || null, + content_hash: contentHash, + hint_md_hash: hintHash, + hint_rule_id: d.rule_id || d.check || null, + confidence: d.confidence ?? null, + proposed_fixes: fixes, + params: Object.keys(diagParams).length > 0 ? diagParams : undefined, + }); + } + } catch { /* best-effort telemetry */ } + } + return result; }; }, diff --git a/src/vendor/d3.v7.min.js b/src/vendor/d3.v7.min.js new file mode 100644 index 0000000..33bb880 --- /dev/null +++ b/src/vendor/d3.v7.min.js @@ -0,0 +1,2 @@ +// https://d3js.org v7.9.0 Copyright 2010-2023 Mike Bostock +!function(t,n){"object"==typeof exports&&"undefined"!=typeof module?n(exports):"function"==typeof define&&define.amd?define(["exports"],n):n((t="undefined"!=typeof globalThis?globalThis:t||self).d3=t.d3||{})}(this,(function(t){"use strict";function n(t,n){return null==t||null==n?NaN:tn?1:t>=n?0:NaN}function e(t,n){return null==t||null==n?NaN:nt?1:n>=t?0:NaN}function r(t){let r,o,a;function u(t,n,e=0,i=t.length){if(e>>1;o(t[r],n)<0?e=r+1:i=r}while(en(t(e),r),a=(n,e)=>t(n)-e):(r=t===n||t===e?t:i,o=t,a=t),{left:u,center:function(t,n,e=0,r=t.length){const i=u(t,n,e,r-1);return i>e&&a(t[i-1],n)>-a(t[i],n)?i-1:i},right:function(t,n,e=0,i=t.length){if(e>>1;o(t[r],n)<=0?e=r+1:i=r}while(e{n(t,e,(r<<=2)+0,(i<<=2)+0,o<<=2),n(t,e,r+1,i+1,o),n(t,e,r+2,i+2,o),n(t,e,r+3,i+3,o)}}));function d(t){return function(n,e,r=e){if(!((e=+e)>=0))throw new RangeError("invalid rx");if(!((r=+r)>=0))throw new RangeError("invalid ry");let{data:i,width:o,height:a}=n;if(!((o=Math.floor(o))>=0))throw new RangeError("invalid width");if(!((a=Math.floor(void 0!==a?a:i.length/o))>=0))throw new RangeError("invalid height");if(!o||!a||!e&&!r)return n;const u=e&&t(e),c=r&&t(r),f=i.slice();return u&&c?(p(u,f,i,o,a),p(u,i,f,o,a),p(u,f,i,o,a),g(c,i,f,o,a),g(c,f,i,o,a),g(c,i,f,o,a)):u?(p(u,i,f,o,a),p(u,f,i,o,a),p(u,i,f,o,a)):c&&(g(c,i,f,o,a),g(c,f,i,o,a),g(c,i,f,o,a)),n}}function p(t,n,e,r,i){for(let o=0,a=r*i;o{if(!((o-=a)>=i))return;let u=t*r[i];const c=a*t;for(let t=i,n=i+c;t{if(!((a-=u)>=o))return;let c=n*i[o];const f=u*n,s=f+u;for(let t=o,n=o+f;t=n&&++e;else{let r=-1;for(let i of t)null!=(i=n(i,++r,t))&&(i=+i)>=i&&++e}return e}function _(t){return 0|t.length}function b(t){return!(t>0)}function m(t){return"object"!=typeof t||"length"in t?t:Array.from(t)}function x(t,n){let e,r=0,i=0,o=0;if(void 0===n)for(let n of t)null!=n&&(n=+n)>=n&&(e=n-i,i+=e/++r,o+=e*(n-i));else{let a=-1;for(let u of t)null!=(u=n(u,++a,t))&&(u=+u)>=u&&(e=u-i,i+=e/++r,o+=e*(u-i))}if(r>1)return o/(r-1)}function w(t,n){const e=x(t,n);return e?Math.sqrt(e):e}function M(t,n){let e,r;if(void 0===n)for(const n of t)null!=n&&(void 0===e?n>=n&&(e=r=n):(e>n&&(e=n),r=o&&(e=r=o):(e>o&&(e=o),r0){for(o=t[--i];i>0&&(n=o,e=t[--i],o=n+e,r=e-(o-n),!r););i>0&&(r<0&&t[i-1]<0||r>0&&t[i-1]>0)&&(e=2*r,n=o+e,e==n-o&&(o=n))}return o}}class InternMap extends Map{constructor(t,n=N){if(super(),Object.defineProperties(this,{_intern:{value:new Map},_key:{value:n}}),null!=t)for(const[n,e]of t)this.set(n,e)}get(t){return super.get(A(this,t))}has(t){return super.has(A(this,t))}set(t,n){return super.set(S(this,t),n)}delete(t){return super.delete(E(this,t))}}class InternSet extends Set{constructor(t,n=N){if(super(),Object.defineProperties(this,{_intern:{value:new Map},_key:{value:n}}),null!=t)for(const n of t)this.add(n)}has(t){return super.has(A(this,t))}add(t){return super.add(S(this,t))}delete(t){return super.delete(E(this,t))}}function A({_intern:t,_key:n},e){const r=n(e);return t.has(r)?t.get(r):e}function S({_intern:t,_key:n},e){const r=n(e);return t.has(r)?t.get(r):(t.set(r,e),e)}function E({_intern:t,_key:n},e){const r=n(e);return t.has(r)&&(e=t.get(r),t.delete(r)),e}function N(t){return null!==t&&"object"==typeof t?t.valueOf():t}function k(t){return t}function C(t,...n){return F(t,k,k,n)}function P(t,...n){return F(t,Array.from,k,n)}function z(t,n){for(let e=1,r=n.length;et.pop().map((([n,e])=>[...t,n,e]))));return t}function $(t,n,...e){return F(t,k,n,e)}function D(t,n,...e){return F(t,Array.from,n,e)}function R(t){if(1!==t.length)throw new Error("duplicate key");return t[0]}function F(t,n,e,r){return function t(i,o){if(o>=r.length)return e(i);const a=new InternMap,u=r[o++];let c=-1;for(const t of i){const n=u(t,++c,i),e=a.get(n);e?e.push(t):a.set(n,[t])}for(const[n,e]of a)a.set(n,t(e,o));return n(a)}(t,0)}function q(t,n){return Array.from(n,(n=>t[n]))}function U(t,...n){if("function"!=typeof t[Symbol.iterator])throw new TypeError("values is not iterable");t=Array.from(t);let[e]=n;if(e&&2!==e.length||n.length>1){const r=Uint32Array.from(t,((t,n)=>n));return n.length>1?(n=n.map((n=>t.map(n))),r.sort(((t,e)=>{for(const r of n){const n=O(r[t],r[e]);if(n)return n}}))):(e=t.map(e),r.sort(((t,n)=>O(e[t],e[n])))),q(t,r)}return t.sort(I(e))}function I(t=n){if(t===n)return O;if("function"!=typeof t)throw new TypeError("compare is not a function");return(n,e)=>{const r=t(n,e);return r||0===r?r:(0===t(e,e))-(0===t(n,n))}}function O(t,n){return(null==t||!(t>=t))-(null==n||!(n>=n))||(tn?1:0)}var B=Array.prototype.slice;function Y(t){return()=>t}const L=Math.sqrt(50),j=Math.sqrt(10),H=Math.sqrt(2);function X(t,n,e){const r=(n-t)/Math.max(0,e),i=Math.floor(Math.log10(r)),o=r/Math.pow(10,i),a=o>=L?10:o>=j?5:o>=H?2:1;let u,c,f;return i<0?(f=Math.pow(10,-i)/a,u=Math.round(t*f),c=Math.round(n*f),u/fn&&--c,f=-f):(f=Math.pow(10,i)*a,u=Math.round(t/f),c=Math.round(n/f),u*fn&&--c),c0))return[];if((t=+t)===(n=+n))return[t];const r=n=i))return[];const u=o-i+1,c=new Array(u);if(r)if(a<0)for(let t=0;t0?(t=Math.floor(t/i)*i,n=Math.ceil(n/i)*i):i<0&&(t=Math.ceil(t*i)/i,n=Math.floor(n*i)/i),r=i}}function K(t){return Math.max(1,Math.ceil(Math.log(v(t))/Math.LN2)+1)}function Q(){var t=k,n=M,e=K;function r(r){Array.isArray(r)||(r=Array.from(r));var i,o,a,u=r.length,c=new Array(u);for(i=0;i=h)if(t>=h&&n===M){const t=V(l,h,e);isFinite(t)&&(t>0?h=(Math.floor(h/t)+1)*t:t<0&&(h=(Math.ceil(h*-t)+1)/-t))}else d.pop()}for(var p=d.length,g=0,y=p;d[g]<=l;)++g;for(;d[y-1]>h;)--y;(g||y0?d[i-1]:l,v.x1=i0)for(i=0;i=n)&&(e=n);else{let r=-1;for(let i of t)null!=(i=n(i,++r,t))&&(e=i)&&(e=i)}return e}function tt(t,n){let e,r=-1,i=-1;if(void 0===n)for(const n of t)++i,null!=n&&(e=n)&&(e=n,r=i);else for(let o of t)null!=(o=n(o,++i,t))&&(e=o)&&(e=o,r=i);return r}function nt(t,n){let e;if(void 0===n)for(const n of t)null!=n&&(e>n||void 0===e&&n>=n)&&(e=n);else{let r=-1;for(let i of t)null!=(i=n(i,++r,t))&&(e>i||void 0===e&&i>=i)&&(e=i)}return e}function et(t,n){let e,r=-1,i=-1;if(void 0===n)for(const n of t)++i,null!=n&&(e>n||void 0===e&&n>=n)&&(e=n,r=i);else for(let o of t)null!=(o=n(o,++i,t))&&(e>o||void 0===e&&o>=o)&&(e=o,r=i);return r}function rt(t,n,e=0,r=1/0,i){if(n=Math.floor(n),e=Math.floor(Math.max(0,e)),r=Math.floor(Math.min(t.length-1,r)),!(e<=n&&n<=r))return t;for(i=void 0===i?O:I(i);r>e;){if(r-e>600){const o=r-e+1,a=n-e+1,u=Math.log(o),c=.5*Math.exp(2*u/3),f=.5*Math.sqrt(u*c*(o-c)/o)*(a-o/2<0?-1:1);rt(t,n,Math.max(e,Math.floor(n-a*c/o+f)),Math.min(r,Math.floor(n+(o-a)*c/o+f)),i)}const o=t[n];let a=e,u=r;for(it(t,e,n),i(t[r],o)>0&&it(t,e,r);a0;)--u}0===i(t[e],o)?it(t,e,u):(++u,it(t,u,r)),u<=n&&(e=u+1),n<=u&&(r=u-1)}return t}function it(t,n,e){const r=t[n];t[n]=t[e],t[e]=r}function ot(t,e=n){let r,i=!1;if(1===e.length){let o;for(const a of t){const t=e(a);(i?n(t,o)>0:0===n(t,t))&&(r=a,o=t,i=!0)}}else for(const n of t)(i?e(n,r)>0:0===e(n,n))&&(r=n,i=!0);return r}function at(t,n,e){if(t=Float64Array.from(function*(t,n){if(void 0===n)for(let n of t)null!=n&&(n=+n)>=n&&(yield n);else{let e=-1;for(let r of t)null!=(r=n(r,++e,t))&&(r=+r)>=r&&(yield r)}}(t,e)),(r=t.length)&&!isNaN(n=+n)){if(n<=0||r<2)return nt(t);if(n>=1)return J(t);var r,i=(r-1)*n,o=Math.floor(i),a=J(rt(t,o).subarray(0,o+1));return a+(nt(t.subarray(o+1))-a)*(i-o)}}function ut(t,n,e=o){if((r=t.length)&&!isNaN(n=+n)){if(n<=0||r<2)return+e(t[0],0,t);if(n>=1)return+e(t[r-1],r-1,t);var r,i=(r-1)*n,a=Math.floor(i),u=+e(t[a],a,t);return u+(+e(t[a+1],a+1,t)-u)*(i-a)}}function ct(t,n,e=o){if(!isNaN(n=+n)){if(r=Float64Array.from(t,((n,r)=>o(e(t[r],r,t)))),n<=0)return et(r);if(n>=1)return tt(r);var r,i=Uint32Array.from(t,((t,n)=>n)),a=r.length-1,u=Math.floor(a*n);return rt(i,u,0,a,((t,n)=>O(r[t],r[n]))),(u=ot(i.subarray(0,u+1),(t=>r[t])))>=0?u:-1}}function ft(t){return Array.from(function*(t){for(const n of t)yield*n}(t))}function st(t,n){return[t,n]}function lt(t,n,e){t=+t,n=+n,e=(i=arguments.length)<2?(n=t,t=0,1):i<3?1:+e;for(var r=-1,i=0|Math.max(0,Math.ceil((n-t)/e)),o=new Array(i);++r+t(n)}function kt(t,n){return n=Math.max(0,t.bandwidth()-2*n)/2,t.round()&&(n=Math.round(n)),e=>+t(e)+n}function Ct(){return!this.__axis}function Pt(t,n){var e=[],r=null,i=null,o=6,a=6,u=3,c="undefined"!=typeof window&&window.devicePixelRatio>1?0:.5,f=t===xt||t===Tt?-1:1,s=t===Tt||t===wt?"x":"y",l=t===xt||t===Mt?St:Et;function h(h){var d=null==r?n.ticks?n.ticks.apply(n,e):n.domain():r,p=null==i?n.tickFormat?n.tickFormat.apply(n,e):mt:i,g=Math.max(o,0)+u,y=n.range(),v=+y[0]+c,_=+y[y.length-1]+c,b=(n.bandwidth?kt:Nt)(n.copy(),c),m=h.selection?h.selection():h,x=m.selectAll(".domain").data([null]),w=m.selectAll(".tick").data(d,n).order(),M=w.exit(),T=w.enter().append("g").attr("class","tick"),A=w.select("line"),S=w.select("text");x=x.merge(x.enter().insert("path",".tick").attr("class","domain").attr("stroke","currentColor")),w=w.merge(T),A=A.merge(T.append("line").attr("stroke","currentColor").attr(s+"2",f*o)),S=S.merge(T.append("text").attr("fill","currentColor").attr(s,f*g).attr("dy",t===xt?"0em":t===Mt?"0.71em":"0.32em")),h!==m&&(x=x.transition(h),w=w.transition(h),A=A.transition(h),S=S.transition(h),M=M.transition(h).attr("opacity",At).attr("transform",(function(t){return isFinite(t=b(t))?l(t+c):this.getAttribute("transform")})),T.attr("opacity",At).attr("transform",(function(t){var n=this.parentNode.__axis;return l((n&&isFinite(n=n(t))?n:b(t))+c)}))),M.remove(),x.attr("d",t===Tt||t===wt?a?"M"+f*a+","+v+"H"+c+"V"+_+"H"+f*a:"M"+c+","+v+"V"+_:a?"M"+v+","+f*a+"V"+c+"H"+_+"V"+f*a:"M"+v+","+c+"H"+_),w.attr("opacity",1).attr("transform",(function(t){return l(b(t)+c)})),A.attr(s+"2",f*o),S.attr(s,f*g).text(p),m.filter(Ct).attr("fill","none").attr("font-size",10).attr("font-family","sans-serif").attr("text-anchor",t===wt?"start":t===Tt?"end":"middle"),m.each((function(){this.__axis=b}))}return h.scale=function(t){return arguments.length?(n=t,h):n},h.ticks=function(){return e=Array.from(arguments),h},h.tickArguments=function(t){return arguments.length?(e=null==t?[]:Array.from(t),h):e.slice()},h.tickValues=function(t){return arguments.length?(r=null==t?null:Array.from(t),h):r&&r.slice()},h.tickFormat=function(t){return arguments.length?(i=t,h):i},h.tickSize=function(t){return arguments.length?(o=a=+t,h):o},h.tickSizeInner=function(t){return arguments.length?(o=+t,h):o},h.tickSizeOuter=function(t){return arguments.length?(a=+t,h):a},h.tickPadding=function(t){return arguments.length?(u=+t,h):u},h.offset=function(t){return arguments.length?(c=+t,h):c},h}var zt={value:()=>{}};function $t(){for(var t,n=0,e=arguments.length,r={};n=0&&(n=t.slice(e+1),t=t.slice(0,e)),t&&!r.hasOwnProperty(t))throw new Error("unknown type: "+t);return{type:t,name:n}}))),a=-1,u=o.length;if(!(arguments.length<2)){if(null!=n&&"function"!=typeof n)throw new Error("invalid callback: "+n);for(;++a0)for(var e,r,i=new Array(e),o=0;o=0&&"xmlns"!==(n=t.slice(0,e))&&(t=t.slice(e+1)),Ut.hasOwnProperty(n)?{space:Ut[n],local:t}:t}function Ot(t){return function(){var n=this.ownerDocument,e=this.namespaceURI;return e===qt&&n.documentElement.namespaceURI===qt?n.createElement(t):n.createElementNS(e,t)}}function Bt(t){return function(){return this.ownerDocument.createElementNS(t.space,t.local)}}function Yt(t){var n=It(t);return(n.local?Bt:Ot)(n)}function Lt(){}function jt(t){return null==t?Lt:function(){return this.querySelector(t)}}function Ht(t){return null==t?[]:Array.isArray(t)?t:Array.from(t)}function Xt(){return[]}function Gt(t){return null==t?Xt:function(){return this.querySelectorAll(t)}}function Vt(t){return function(){return this.matches(t)}}function Wt(t){return function(n){return n.matches(t)}}var Zt=Array.prototype.find;function Kt(){return this.firstElementChild}var Qt=Array.prototype.filter;function Jt(){return Array.from(this.children)}function tn(t){return new Array(t.length)}function nn(t,n){this.ownerDocument=t.ownerDocument,this.namespaceURI=t.namespaceURI,this._next=null,this._parent=t,this.__data__=n}function en(t,n,e,r,i,o){for(var a,u=0,c=n.length,f=o.length;un?1:t>=n?0:NaN}function cn(t){return function(){this.removeAttribute(t)}}function fn(t){return function(){this.removeAttributeNS(t.space,t.local)}}function sn(t,n){return function(){this.setAttribute(t,n)}}function ln(t,n){return function(){this.setAttributeNS(t.space,t.local,n)}}function hn(t,n){return function(){var e=n.apply(this,arguments);null==e?this.removeAttribute(t):this.setAttribute(t,e)}}function dn(t,n){return function(){var e=n.apply(this,arguments);null==e?this.removeAttributeNS(t.space,t.local):this.setAttributeNS(t.space,t.local,e)}}function pn(t){return t.ownerDocument&&t.ownerDocument.defaultView||t.document&&t||t.defaultView}function gn(t){return function(){this.style.removeProperty(t)}}function yn(t,n,e){return function(){this.style.setProperty(t,n,e)}}function vn(t,n,e){return function(){var r=n.apply(this,arguments);null==r?this.style.removeProperty(t):this.style.setProperty(t,r,e)}}function _n(t,n){return t.style.getPropertyValue(n)||pn(t).getComputedStyle(t,null).getPropertyValue(n)}function bn(t){return function(){delete this[t]}}function mn(t,n){return function(){this[t]=n}}function xn(t,n){return function(){var e=n.apply(this,arguments);null==e?delete this[t]:this[t]=e}}function wn(t){return t.trim().split(/^|\s+/)}function Mn(t){return t.classList||new Tn(t)}function Tn(t){this._node=t,this._names=wn(t.getAttribute("class")||"")}function An(t,n){for(var e=Mn(t),r=-1,i=n.length;++r=0&&(this._names.splice(n,1),this._node.setAttribute("class",this._names.join(" ")))},contains:function(t){return this._names.indexOf(t)>=0}};var Gn=[null];function Vn(t,n){this._groups=t,this._parents=n}function Wn(){return new Vn([[document.documentElement]],Gn)}function Zn(t){return"string"==typeof t?new Vn([[document.querySelector(t)]],[document.documentElement]):new Vn([[t]],Gn)}Vn.prototype=Wn.prototype={constructor:Vn,select:function(t){"function"!=typeof t&&(t=jt(t));for(var n=this._groups,e=n.length,r=new Array(e),i=0;i=m&&(m=b+1);!(_=y[m])&&++m=0;)(r=i[o])&&(a&&4^r.compareDocumentPosition(a)&&a.parentNode.insertBefore(r,a),a=r);return this},sort:function(t){function n(n,e){return n&&e?t(n.__data__,e.__data__):!n-!e}t||(t=un);for(var e=this._groups,r=e.length,i=new Array(r),o=0;o1?this.each((null==n?gn:"function"==typeof n?vn:yn)(t,n,null==e?"":e)):_n(this.node(),t)},property:function(t,n){return arguments.length>1?this.each((null==n?bn:"function"==typeof n?xn:mn)(t,n)):this.node()[t]},classed:function(t,n){var e=wn(t+"");if(arguments.length<2){for(var r=Mn(this.node()),i=-1,o=e.length;++i=0&&(n=t.slice(e+1),t=t.slice(0,e)),{type:t,name:n}}))}(t+""),a=o.length;if(!(arguments.length<2)){for(u=n?Ln:Yn,r=0;r()=>t;function fe(t,{sourceEvent:n,subject:e,target:r,identifier:i,active:o,x:a,y:u,dx:c,dy:f,dispatch:s}){Object.defineProperties(this,{type:{value:t,enumerable:!0,configurable:!0},sourceEvent:{value:n,enumerable:!0,configurable:!0},subject:{value:e,enumerable:!0,configurable:!0},target:{value:r,enumerable:!0,configurable:!0},identifier:{value:i,enumerable:!0,configurable:!0},active:{value:o,enumerable:!0,configurable:!0},x:{value:a,enumerable:!0,configurable:!0},y:{value:u,enumerable:!0,configurable:!0},dx:{value:c,enumerable:!0,configurable:!0},dy:{value:f,enumerable:!0,configurable:!0},_:{value:s}})}function se(t){return!t.ctrlKey&&!t.button}function le(){return this.parentNode}function he(t,n){return null==n?{x:t.x,y:t.y}:n}function de(){return navigator.maxTouchPoints||"ontouchstart"in this}function pe(t,n,e){t.prototype=n.prototype=e,e.constructor=t}function ge(t,n){var e=Object.create(t.prototype);for(var r in n)e[r]=n[r];return e}function ye(){}fe.prototype.on=function(){var t=this._.on.apply(this._,arguments);return t===this._?this:t};var ve=.7,_e=1/ve,be="\\s*([+-]?\\d+)\\s*",me="\\s*([+-]?(?:\\d*\\.)?\\d+(?:[eE][+-]?\\d+)?)\\s*",xe="\\s*([+-]?(?:\\d*\\.)?\\d+(?:[eE][+-]?\\d+)?)%\\s*",we=/^#([0-9a-f]{3,8})$/,Me=new RegExp(`^rgb\\(${be},${be},${be}\\)$`),Te=new RegExp(`^rgb\\(${xe},${xe},${xe}\\)$`),Ae=new RegExp(`^rgba\\(${be},${be},${be},${me}\\)$`),Se=new RegExp(`^rgba\\(${xe},${xe},${xe},${me}\\)$`),Ee=new RegExp(`^hsl\\(${me},${xe},${xe}\\)$`),Ne=new RegExp(`^hsla\\(${me},${xe},${xe},${me}\\)$`),ke={aliceblue:15792383,antiquewhite:16444375,aqua:65535,aquamarine:8388564,azure:15794175,beige:16119260,bisque:16770244,black:0,blanchedalmond:16772045,blue:255,blueviolet:9055202,brown:10824234,burlywood:14596231,cadetblue:6266528,chartreuse:8388352,chocolate:13789470,coral:16744272,cornflowerblue:6591981,cornsilk:16775388,crimson:14423100,cyan:65535,darkblue:139,darkcyan:35723,darkgoldenrod:12092939,darkgray:11119017,darkgreen:25600,darkgrey:11119017,darkkhaki:12433259,darkmagenta:9109643,darkolivegreen:5597999,darkorange:16747520,darkorchid:10040012,darkred:9109504,darksalmon:15308410,darkseagreen:9419919,darkslateblue:4734347,darkslategray:3100495,darkslategrey:3100495,darkturquoise:52945,darkviolet:9699539,deeppink:16716947,deepskyblue:49151,dimgray:6908265,dimgrey:6908265,dodgerblue:2003199,firebrick:11674146,floralwhite:16775920,forestgreen:2263842,fuchsia:16711935,gainsboro:14474460,ghostwhite:16316671,gold:16766720,goldenrod:14329120,gray:8421504,green:32768,greenyellow:11403055,grey:8421504,honeydew:15794160,hotpink:16738740,indianred:13458524,indigo:4915330,ivory:16777200,khaki:15787660,lavender:15132410,lavenderblush:16773365,lawngreen:8190976,lemonchiffon:16775885,lightblue:11393254,lightcoral:15761536,lightcyan:14745599,lightgoldenrodyellow:16448210,lightgray:13882323,lightgreen:9498256,lightgrey:13882323,lightpink:16758465,lightsalmon:16752762,lightseagreen:2142890,lightskyblue:8900346,lightslategray:7833753,lightslategrey:7833753,lightsteelblue:11584734,lightyellow:16777184,lime:65280,limegreen:3329330,linen:16445670,magenta:16711935,maroon:8388608,mediumaquamarine:6737322,mediumblue:205,mediumorchid:12211667,mediumpurple:9662683,mediumseagreen:3978097,mediumslateblue:8087790,mediumspringgreen:64154,mediumturquoise:4772300,mediumvioletred:13047173,midnightblue:1644912,mintcream:16121850,mistyrose:16770273,moccasin:16770229,navajowhite:16768685,navy:128,oldlace:16643558,olive:8421376,olivedrab:7048739,orange:16753920,orangered:16729344,orchid:14315734,palegoldenrod:15657130,palegreen:10025880,paleturquoise:11529966,palevioletred:14381203,papayawhip:16773077,peachpuff:16767673,peru:13468991,pink:16761035,plum:14524637,powderblue:11591910,purple:8388736,rebeccapurple:6697881,red:16711680,rosybrown:12357519,royalblue:4286945,saddlebrown:9127187,salmon:16416882,sandybrown:16032864,seagreen:3050327,seashell:16774638,sienna:10506797,silver:12632256,skyblue:8900331,slateblue:6970061,slategray:7372944,slategrey:7372944,snow:16775930,springgreen:65407,steelblue:4620980,tan:13808780,teal:32896,thistle:14204888,tomato:16737095,turquoise:4251856,violet:15631086,wheat:16113331,white:16777215,whitesmoke:16119285,yellow:16776960,yellowgreen:10145074};function Ce(){return this.rgb().formatHex()}function Pe(){return this.rgb().formatRgb()}function ze(t){var n,e;return t=(t+"").trim().toLowerCase(),(n=we.exec(t))?(e=n[1].length,n=parseInt(n[1],16),6===e?$e(n):3===e?new qe(n>>8&15|n>>4&240,n>>4&15|240&n,(15&n)<<4|15&n,1):8===e?De(n>>24&255,n>>16&255,n>>8&255,(255&n)/255):4===e?De(n>>12&15|n>>8&240,n>>8&15|n>>4&240,n>>4&15|240&n,((15&n)<<4|15&n)/255):null):(n=Me.exec(t))?new qe(n[1],n[2],n[3],1):(n=Te.exec(t))?new qe(255*n[1]/100,255*n[2]/100,255*n[3]/100,1):(n=Ae.exec(t))?De(n[1],n[2],n[3],n[4]):(n=Se.exec(t))?De(255*n[1]/100,255*n[2]/100,255*n[3]/100,n[4]):(n=Ee.exec(t))?Le(n[1],n[2]/100,n[3]/100,1):(n=Ne.exec(t))?Le(n[1],n[2]/100,n[3]/100,n[4]):ke.hasOwnProperty(t)?$e(ke[t]):"transparent"===t?new qe(NaN,NaN,NaN,0):null}function $e(t){return new qe(t>>16&255,t>>8&255,255&t,1)}function De(t,n,e,r){return r<=0&&(t=n=e=NaN),new qe(t,n,e,r)}function Re(t){return t instanceof ye||(t=ze(t)),t?new qe((t=t.rgb()).r,t.g,t.b,t.opacity):new qe}function Fe(t,n,e,r){return 1===arguments.length?Re(t):new qe(t,n,e,null==r?1:r)}function qe(t,n,e,r){this.r=+t,this.g=+n,this.b=+e,this.opacity=+r}function Ue(){return`#${Ye(this.r)}${Ye(this.g)}${Ye(this.b)}`}function Ie(){const t=Oe(this.opacity);return`${1===t?"rgb(":"rgba("}${Be(this.r)}, ${Be(this.g)}, ${Be(this.b)}${1===t?")":`, ${t})`}`}function Oe(t){return isNaN(t)?1:Math.max(0,Math.min(1,t))}function Be(t){return Math.max(0,Math.min(255,Math.round(t)||0))}function Ye(t){return((t=Be(t))<16?"0":"")+t.toString(16)}function Le(t,n,e,r){return r<=0?t=n=e=NaN:e<=0||e>=1?t=n=NaN:n<=0&&(t=NaN),new Xe(t,n,e,r)}function je(t){if(t instanceof Xe)return new Xe(t.h,t.s,t.l,t.opacity);if(t instanceof ye||(t=ze(t)),!t)return new Xe;if(t instanceof Xe)return t;var n=(t=t.rgb()).r/255,e=t.g/255,r=t.b/255,i=Math.min(n,e,r),o=Math.max(n,e,r),a=NaN,u=o-i,c=(o+i)/2;return u?(a=n===o?(e-r)/u+6*(e0&&c<1?0:a,new Xe(a,u,c,t.opacity)}function He(t,n,e,r){return 1===arguments.length?je(t):new Xe(t,n,e,null==r?1:r)}function Xe(t,n,e,r){this.h=+t,this.s=+n,this.l=+e,this.opacity=+r}function Ge(t){return(t=(t||0)%360)<0?t+360:t}function Ve(t){return Math.max(0,Math.min(1,t||0))}function We(t,n,e){return 255*(t<60?n+(e-n)*t/60:t<180?e:t<240?n+(e-n)*(240-t)/60:n)}pe(ye,ze,{copy(t){return Object.assign(new this.constructor,this,t)},displayable(){return this.rgb().displayable()},hex:Ce,formatHex:Ce,formatHex8:function(){return this.rgb().formatHex8()},formatHsl:function(){return je(this).formatHsl()},formatRgb:Pe,toString:Pe}),pe(qe,Fe,ge(ye,{brighter(t){return t=null==t?_e:Math.pow(_e,t),new qe(this.r*t,this.g*t,this.b*t,this.opacity)},darker(t){return t=null==t?ve:Math.pow(ve,t),new qe(this.r*t,this.g*t,this.b*t,this.opacity)},rgb(){return this},clamp(){return new qe(Be(this.r),Be(this.g),Be(this.b),Oe(this.opacity))},displayable(){return-.5<=this.r&&this.r<255.5&&-.5<=this.g&&this.g<255.5&&-.5<=this.b&&this.b<255.5&&0<=this.opacity&&this.opacity<=1},hex:Ue,formatHex:Ue,formatHex8:function(){return`#${Ye(this.r)}${Ye(this.g)}${Ye(this.b)}${Ye(255*(isNaN(this.opacity)?1:this.opacity))}`},formatRgb:Ie,toString:Ie})),pe(Xe,He,ge(ye,{brighter(t){return t=null==t?_e:Math.pow(_e,t),new Xe(this.h,this.s,this.l*t,this.opacity)},darker(t){return t=null==t?ve:Math.pow(ve,t),new Xe(this.h,this.s,this.l*t,this.opacity)},rgb(){var t=this.h%360+360*(this.h<0),n=isNaN(t)||isNaN(this.s)?0:this.s,e=this.l,r=e+(e<.5?e:1-e)*n,i=2*e-r;return new qe(We(t>=240?t-240:t+120,i,r),We(t,i,r),We(t<120?t+240:t-120,i,r),this.opacity)},clamp(){return new Xe(Ge(this.h),Ve(this.s),Ve(this.l),Oe(this.opacity))},displayable(){return(0<=this.s&&this.s<=1||isNaN(this.s))&&0<=this.l&&this.l<=1&&0<=this.opacity&&this.opacity<=1},formatHsl(){const t=Oe(this.opacity);return`${1===t?"hsl(":"hsla("}${Ge(this.h)}, ${100*Ve(this.s)}%, ${100*Ve(this.l)}%${1===t?")":`, ${t})`}`}}));const Ze=Math.PI/180,Ke=180/Math.PI,Qe=.96422,Je=1,tr=.82521,nr=4/29,er=6/29,rr=3*er*er,ir=er*er*er;function or(t){if(t instanceof ur)return new ur(t.l,t.a,t.b,t.opacity);if(t instanceof pr)return gr(t);t instanceof qe||(t=Re(t));var n,e,r=lr(t.r),i=lr(t.g),o=lr(t.b),a=cr((.2225045*r+.7168786*i+.0606169*o)/Je);return r===i&&i===o?n=e=a:(n=cr((.4360747*r+.3850649*i+.1430804*o)/Qe),e=cr((.0139322*r+.0971045*i+.7141733*o)/tr)),new ur(116*a-16,500*(n-a),200*(a-e),t.opacity)}function ar(t,n,e,r){return 1===arguments.length?or(t):new ur(t,n,e,null==r?1:r)}function ur(t,n,e,r){this.l=+t,this.a=+n,this.b=+e,this.opacity=+r}function cr(t){return t>ir?Math.pow(t,1/3):t/rr+nr}function fr(t){return t>er?t*t*t:rr*(t-nr)}function sr(t){return 255*(t<=.0031308?12.92*t:1.055*Math.pow(t,1/2.4)-.055)}function lr(t){return(t/=255)<=.04045?t/12.92:Math.pow((t+.055)/1.055,2.4)}function hr(t){if(t instanceof pr)return new pr(t.h,t.c,t.l,t.opacity);if(t instanceof ur||(t=or(t)),0===t.a&&0===t.b)return new pr(NaN,0=1?(e=1,n-1):Math.floor(e*n),i=t[r],o=t[r+1],a=r>0?t[r-1]:2*i-o,u=r()=>t;function Cr(t,n){return function(e){return t+e*n}}function Pr(t,n){var e=n-t;return e?Cr(t,e>180||e<-180?e-360*Math.round(e/360):e):kr(isNaN(t)?n:t)}function zr(t){return 1==(t=+t)?$r:function(n,e){return e-n?function(t,n,e){return t=Math.pow(t,e),n=Math.pow(n,e)-t,e=1/e,function(r){return Math.pow(t+r*n,e)}}(n,e,t):kr(isNaN(n)?e:n)}}function $r(t,n){var e=n-t;return e?Cr(t,e):kr(isNaN(t)?n:t)}var Dr=function t(n){var e=zr(n);function r(t,n){var r=e((t=Fe(t)).r,(n=Fe(n)).r),i=e(t.g,n.g),o=e(t.b,n.b),a=$r(t.opacity,n.opacity);return function(n){return t.r=r(n),t.g=i(n),t.b=o(n),t.opacity=a(n),t+""}}return r.gamma=t,r}(1);function Rr(t){return function(n){var e,r,i=n.length,o=new Array(i),a=new Array(i),u=new Array(i);for(e=0;eo&&(i=n.slice(o,i),u[a]?u[a]+=i:u[++a]=i),(e=e[0])===(r=r[0])?u[a]?u[a]+=r:u[++a]=r:(u[++a]=null,c.push({i:a,x:Yr(e,r)})),o=Hr.lastIndex;return o180?n+=360:n-t>180&&(t+=360),o.push({i:e.push(i(e)+"rotate(",null,r)-2,x:Yr(t,n)})):n&&e.push(i(e)+"rotate("+n+r)}(o.rotate,a.rotate,u,c),function(t,n,e,o){t!==n?o.push({i:e.push(i(e)+"skewX(",null,r)-2,x:Yr(t,n)}):n&&e.push(i(e)+"skewX("+n+r)}(o.skewX,a.skewX,u,c),function(t,n,e,r,o,a){if(t!==e||n!==r){var u=o.push(i(o)+"scale(",null,",",null,")");a.push({i:u-4,x:Yr(t,e)},{i:u-2,x:Yr(n,r)})}else 1===e&&1===r||o.push(i(o)+"scale("+e+","+r+")")}(o.scaleX,o.scaleY,a.scaleX,a.scaleY,u,c),o=a=null,function(t){for(var n,e=-1,r=c.length;++e=0&&n._call.call(void 0,t),n=n._next;--yi}function Ci(){xi=(mi=Mi.now())+wi,yi=vi=0;try{ki()}finally{yi=0,function(){var t,n,e=pi,r=1/0;for(;e;)e._call?(r>e._time&&(r=e._time),t=e,e=e._next):(n=e._next,e._next=null,e=t?t._next=n:pi=n);gi=t,zi(r)}(),xi=0}}function Pi(){var t=Mi.now(),n=t-mi;n>bi&&(wi-=n,mi=t)}function zi(t){yi||(vi&&(vi=clearTimeout(vi)),t-xi>24?(t<1/0&&(vi=setTimeout(Ci,t-Mi.now()-wi)),_i&&(_i=clearInterval(_i))):(_i||(mi=Mi.now(),_i=setInterval(Pi,bi)),yi=1,Ti(Ci)))}function $i(t,n,e){var r=new Ei;return n=null==n?0:+n,r.restart((e=>{r.stop(),t(e+n)}),n,e),r}Ei.prototype=Ni.prototype={constructor:Ei,restart:function(t,n,e){if("function"!=typeof t)throw new TypeError("callback is not a function");e=(null==e?Ai():+e)+(null==n?0:+n),this._next||gi===this||(gi?gi._next=this:pi=this,gi=this),this._call=t,this._time=e,zi()},stop:function(){this._call&&(this._call=null,this._time=1/0,zi())}};var Di=$t("start","end","cancel","interrupt"),Ri=[],Fi=0,qi=1,Ui=2,Ii=3,Oi=4,Bi=5,Yi=6;function Li(t,n,e,r,i,o){var a=t.__transition;if(a){if(e in a)return}else t.__transition={};!function(t,n,e){var r,i=t.__transition;function o(t){e.state=qi,e.timer.restart(a,e.delay,e.time),e.delay<=t&&a(t-e.delay)}function a(o){var f,s,l,h;if(e.state!==qi)return c();for(f in i)if((h=i[f]).name===e.name){if(h.state===Ii)return $i(a);h.state===Oi?(h.state=Yi,h.timer.stop(),h.on.call("interrupt",t,t.__data__,h.index,h.group),delete i[f]):+fFi)throw new Error("too late; already scheduled");return e}function Hi(t,n){var e=Xi(t,n);if(e.state>Ii)throw new Error("too late; already running");return e}function Xi(t,n){var e=t.__transition;if(!e||!(e=e[n]))throw new Error("transition not found");return e}function Gi(t,n){var e,r,i,o=t.__transition,a=!0;if(o){for(i in n=null==n?null:n+"",o)(e=o[i]).name===n?(r=e.state>Ui&&e.state=0&&(t=t.slice(0,n)),!t||"start"===t}))}(n)?ji:Hi;return function(){var a=o(this,t),u=a.on;u!==r&&(i=(r=u).copy()).on(n,e),a.on=i}}(e,t,n))},attr:function(t,n){var e=It(t),r="transform"===e?ni:Ki;return this.attrTween(t,"function"==typeof n?(e.local?ro:eo)(e,r,Zi(this,"attr."+t,n)):null==n?(e.local?Ji:Qi)(e):(e.local?no:to)(e,r,n))},attrTween:function(t,n){var e="attr."+t;if(arguments.length<2)return(e=this.tween(e))&&e._value;if(null==n)return this.tween(e,null);if("function"!=typeof n)throw new Error;var r=It(t);return this.tween(e,(r.local?io:oo)(r,n))},style:function(t,n,e){var r="transform"==(t+="")?ti:Ki;return null==n?this.styleTween(t,function(t,n){var e,r,i;return function(){var o=_n(this,t),a=(this.style.removeProperty(t),_n(this,t));return o===a?null:o===e&&a===r?i:i=n(e=o,r=a)}}(t,r)).on("end.style."+t,lo(t)):"function"==typeof n?this.styleTween(t,function(t,n,e){var r,i,o;return function(){var a=_n(this,t),u=e(this),c=u+"";return null==u&&(this.style.removeProperty(t),c=u=_n(this,t)),a===c?null:a===r&&c===i?o:(i=c,o=n(r=a,u))}}(t,r,Zi(this,"style."+t,n))).each(function(t,n){var e,r,i,o,a="style."+n,u="end."+a;return function(){var c=Hi(this,t),f=c.on,s=null==c.value[a]?o||(o=lo(n)):void 0;f===e&&i===s||(r=(e=f).copy()).on(u,i=s),c.on=r}}(this._id,t)):this.styleTween(t,function(t,n,e){var r,i,o=e+"";return function(){var a=_n(this,t);return a===o?null:a===r?i:i=n(r=a,e)}}(t,r,n),e).on("end.style."+t,null)},styleTween:function(t,n,e){var r="style."+(t+="");if(arguments.length<2)return(r=this.tween(r))&&r._value;if(null==n)return this.tween(r,null);if("function"!=typeof n)throw new Error;return this.tween(r,function(t,n,e){var r,i;function o(){var o=n.apply(this,arguments);return o!==i&&(r=(i=o)&&function(t,n,e){return function(r){this.style.setProperty(t,n.call(this,r),e)}}(t,o,e)),r}return o._value=n,o}(t,n,null==e?"":e))},text:function(t){return this.tween("text","function"==typeof t?function(t){return function(){var n=t(this);this.textContent=null==n?"":n}}(Zi(this,"text",t)):function(t){return function(){this.textContent=t}}(null==t?"":t+""))},textTween:function(t){var n="text";if(arguments.length<1)return(n=this.tween(n))&&n._value;if(null==t)return this.tween(n,null);if("function"!=typeof t)throw new Error;return this.tween(n,function(t){var n,e;function r(){var r=t.apply(this,arguments);return r!==e&&(n=(e=r)&&function(t){return function(n){this.textContent=t.call(this,n)}}(r)),n}return r._value=t,r}(t))},remove:function(){return this.on("end.remove",function(t){return function(){var n=this.parentNode;for(var e in this.__transition)if(+e!==t)return;n&&n.removeChild(this)}}(this._id))},tween:function(t,n){var e=this._id;if(t+="",arguments.length<2){for(var r,i=Xi(this.node(),e).tween,o=0,a=i.length;o()=>t;function Qo(t,{sourceEvent:n,target:e,selection:r,mode:i,dispatch:o}){Object.defineProperties(this,{type:{value:t,enumerable:!0,configurable:!0},sourceEvent:{value:n,enumerable:!0,configurable:!0},target:{value:e,enumerable:!0,configurable:!0},selection:{value:r,enumerable:!0,configurable:!0},mode:{value:i,enumerable:!0,configurable:!0},_:{value:o}})}function Jo(t){t.preventDefault(),t.stopImmediatePropagation()}var ta={name:"drag"},na={name:"space"},ea={name:"handle"},ra={name:"center"};const{abs:ia,max:oa,min:aa}=Math;function ua(t){return[+t[0],+t[1]]}function ca(t){return[ua(t[0]),ua(t[1])]}var fa={name:"x",handles:["w","e"].map(va),input:function(t,n){return null==t?null:[[+t[0],n[0][1]],[+t[1],n[1][1]]]},output:function(t){return t&&[t[0][0],t[1][0]]}},sa={name:"y",handles:["n","s"].map(va),input:function(t,n){return null==t?null:[[n[0][0],+t[0]],[n[1][0],+t[1]]]},output:function(t){return t&&[t[0][1],t[1][1]]}},la={name:"xy",handles:["n","w","e","s","nw","ne","sw","se"].map(va),input:function(t){return null==t?null:ca(t)},output:function(t){return t}},ha={overlay:"crosshair",selection:"move",n:"ns-resize",e:"ew-resize",s:"ns-resize",w:"ew-resize",nw:"nwse-resize",ne:"nesw-resize",se:"nwse-resize",sw:"nesw-resize"},da={e:"w",w:"e",nw:"ne",ne:"nw",se:"sw",sw:"se"},pa={n:"s",s:"n",nw:"sw",ne:"se",se:"ne",sw:"nw"},ga={overlay:1,selection:1,n:null,e:1,s:null,w:-1,nw:-1,ne:1,se:1,sw:-1},ya={overlay:1,selection:1,n:-1,e:null,s:1,w:null,nw:-1,ne:-1,se:1,sw:1};function va(t){return{type:t}}function _a(t){return!t.ctrlKey&&!t.button}function ba(){var t=this.ownerSVGElement||this;return t.hasAttribute("viewBox")?[[(t=t.viewBox.baseVal).x,t.y],[t.x+t.width,t.y+t.height]]:[[0,0],[t.width.baseVal.value,t.height.baseVal.value]]}function ma(){return navigator.maxTouchPoints||"ontouchstart"in this}function xa(t){for(;!t.__brush;)if(!(t=t.parentNode))return;return t.__brush}function wa(t){var n,e=ba,r=_a,i=ma,o=!0,a=$t("start","brush","end"),u=6;function c(n){var e=n.property("__brush",g).selectAll(".overlay").data([va("overlay")]);e.enter().append("rect").attr("class","overlay").attr("pointer-events","all").attr("cursor",ha.overlay).merge(e).each((function(){var t=xa(this).extent;Zn(this).attr("x",t[0][0]).attr("y",t[0][1]).attr("width",t[1][0]-t[0][0]).attr("height",t[1][1]-t[0][1])})),n.selectAll(".selection").data([va("selection")]).enter().append("rect").attr("class","selection").attr("cursor",ha.selection).attr("fill","#777").attr("fill-opacity",.3).attr("stroke","#fff").attr("shape-rendering","crispEdges");var r=n.selectAll(".handle").data(t.handles,(function(t){return t.type}));r.exit().remove(),r.enter().append("rect").attr("class",(function(t){return"handle handle--"+t.type})).attr("cursor",(function(t){return ha[t.type]})),n.each(f).attr("fill","none").attr("pointer-events","all").on("mousedown.brush",h).filter(i).on("touchstart.brush",h).on("touchmove.brush",d).on("touchend.brush touchcancel.brush",p).style("touch-action","none").style("-webkit-tap-highlight-color","rgba(0,0,0,0)")}function f(){var t=Zn(this),n=xa(this).selection;n?(t.selectAll(".selection").style("display",null).attr("x",n[0][0]).attr("y",n[0][1]).attr("width",n[1][0]-n[0][0]).attr("height",n[1][1]-n[0][1]),t.selectAll(".handle").style("display",null).attr("x",(function(t){return"e"===t.type[t.type.length-1]?n[1][0]-u/2:n[0][0]-u/2})).attr("y",(function(t){return"s"===t.type[0]?n[1][1]-u/2:n[0][1]-u/2})).attr("width",(function(t){return"n"===t.type||"s"===t.type?n[1][0]-n[0][0]+u:u})).attr("height",(function(t){return"e"===t.type||"w"===t.type?n[1][1]-n[0][1]+u:u}))):t.selectAll(".selection,.handle").style("display","none").attr("x",null).attr("y",null).attr("width",null).attr("height",null)}function s(t,n,e){var r=t.__brush.emitter;return!r||e&&r.clean?new l(t,n,e):r}function l(t,n,e){this.that=t,this.args=n,this.state=t.__brush,this.active=0,this.clean=e}function h(e){if((!n||e.touches)&&r.apply(this,arguments)){var i,a,u,c,l,h,d,p,g,y,v,_=this,b=e.target.__data__.type,m="selection"===(o&&e.metaKey?b="overlay":b)?ta:o&&e.altKey?ra:ea,x=t===sa?null:ga[b],w=t===fa?null:ya[b],M=xa(_),T=M.extent,A=M.selection,S=T[0][0],E=T[0][1],N=T[1][0],k=T[1][1],C=0,P=0,z=x&&w&&o&&e.shiftKey,$=Array.from(e.touches||[e],(t=>{const n=t.identifier;return(t=ne(t,_)).point0=t.slice(),t.identifier=n,t}));Gi(_);var D=s(_,arguments,!0).beforestart();if("overlay"===b){A&&(g=!0);const n=[$[0],$[1]||$[0]];M.selection=A=[[i=t===sa?S:aa(n[0][0],n[1][0]),u=t===fa?E:aa(n[0][1],n[1][1])],[l=t===sa?N:oa(n[0][0],n[1][0]),d=t===fa?k:oa(n[0][1],n[1][1])]],$.length>1&&I(e)}else i=A[0][0],u=A[0][1],l=A[1][0],d=A[1][1];a=i,c=u,h=l,p=d;var R=Zn(_).attr("pointer-events","none"),F=R.selectAll(".overlay").attr("cursor",ha[b]);if(e.touches)D.moved=U,D.ended=O;else{var q=Zn(e.view).on("mousemove.brush",U,!0).on("mouseup.brush",O,!0);o&&q.on("keydown.brush",(function(t){switch(t.keyCode){case 16:z=x&&w;break;case 18:m===ea&&(x&&(l=h-C*x,i=a+C*x),w&&(d=p-P*w,u=c+P*w),m=ra,I(t));break;case 32:m!==ea&&m!==ra||(x<0?l=h-C:x>0&&(i=a-C),w<0?d=p-P:w>0&&(u=c-P),m=na,F.attr("cursor",ha.selection),I(t));break;default:return}Jo(t)}),!0).on("keyup.brush",(function(t){switch(t.keyCode){case 16:z&&(y=v=z=!1,I(t));break;case 18:m===ra&&(x<0?l=h:x>0&&(i=a),w<0?d=p:w>0&&(u=c),m=ea,I(t));break;case 32:m===na&&(t.altKey?(x&&(l=h-C*x,i=a+C*x),w&&(d=p-P*w,u=c+P*w),m=ra):(x<0?l=h:x>0&&(i=a),w<0?d=p:w>0&&(u=c),m=ea),F.attr("cursor",ha[b]),I(t));break;default:return}Jo(t)}),!0),ae(e.view)}f.call(_),D.start(e,m.name)}function U(t){for(const n of t.changedTouches||[t])for(const t of $)t.identifier===n.identifier&&(t.cur=ne(n,_));if(z&&!y&&!v&&1===$.length){const t=$[0];ia(t.cur[0]-t[0])>ia(t.cur[1]-t[1])?v=!0:y=!0}for(const t of $)t.cur&&(t[0]=t.cur[0],t[1]=t.cur[1]);g=!0,Jo(t),I(t)}function I(t){const n=$[0],e=n.point0;var r;switch(C=n[0]-e[0],P=n[1]-e[1],m){case na:case ta:x&&(C=oa(S-i,aa(N-l,C)),a=i+C,h=l+C),w&&(P=oa(E-u,aa(k-d,P)),c=u+P,p=d+P);break;case ea:$[1]?(x&&(a=oa(S,aa(N,$[0][0])),h=oa(S,aa(N,$[1][0])),x=1),w&&(c=oa(E,aa(k,$[0][1])),p=oa(E,aa(k,$[1][1])),w=1)):(x<0?(C=oa(S-i,aa(N-i,C)),a=i+C,h=l):x>0&&(C=oa(S-l,aa(N-l,C)),a=i,h=l+C),w<0?(P=oa(E-u,aa(k-u,P)),c=u+P,p=d):w>0&&(P=oa(E-d,aa(k-d,P)),c=u,p=d+P));break;case ra:x&&(a=oa(S,aa(N,i-C*x)),h=oa(S,aa(N,l+C*x))),w&&(c=oa(E,aa(k,u-P*w)),p=oa(E,aa(k,d+P*w)))}ht+e))}function za(t,n){var e=0,r=null,i=null,o=null;function a(a){var u,c=a.length,f=new Array(c),s=Pa(0,c),l=new Array(c*c),h=new Array(c),d=0;a=Float64Array.from({length:c*c},n?(t,n)=>a[n%c][n/c|0]:(t,n)=>a[n/c|0][n%c]);for(let n=0;nr(f[t],f[n])));for(const e of s){const r=n;if(t){const t=Pa(1+~c,c).filter((t=>t<0?a[~t*c+e]:a[e*c+t]));i&&t.sort(((t,n)=>i(t<0?-a[~t*c+e]:a[e*c+t],n<0?-a[~n*c+e]:a[e*c+n])));for(const r of t)if(r<0){(l[~r*c+e]||(l[~r*c+e]={source:null,target:null})).target={index:e,startAngle:n,endAngle:n+=a[~r*c+e]*d,value:a[~r*c+e]}}else{(l[e*c+r]||(l[e*c+r]={source:null,target:null})).source={index:e,startAngle:n,endAngle:n+=a[e*c+r]*d,value:a[e*c+r]}}h[e]={index:e,startAngle:r,endAngle:n,value:f[e]}}else{const t=Pa(0,c).filter((t=>a[e*c+t]||a[t*c+e]));i&&t.sort(((t,n)=>i(a[e*c+t],a[e*c+n])));for(const r of t){let t;if(e=0))throw new Error(`invalid digits: ${t}`);if(n>15)return qa;const e=10**n;return function(t){this._+=t[0];for(let n=1,r=t.length;nRa)if(Math.abs(s*u-c*f)>Ra&&i){let h=e-o,d=r-a,p=u*u+c*c,g=h*h+d*d,y=Math.sqrt(p),v=Math.sqrt(l),_=i*Math.tan(($a-Math.acos((p+l-g)/(2*y*v)))/2),b=_/v,m=_/y;Math.abs(b-1)>Ra&&this._append`L${t+b*f},${n+b*s}`,this._append`A${i},${i},0,0,${+(s*h>f*d)},${this._x1=t+m*u},${this._y1=n+m*c}`}else this._append`L${this._x1=t},${this._y1=n}`;else;}arc(t,n,e,r,i,o){if(t=+t,n=+n,o=!!o,(e=+e)<0)throw new Error(`negative radius: ${e}`);let a=e*Math.cos(r),u=e*Math.sin(r),c=t+a,f=n+u,s=1^o,l=o?r-i:i-r;null===this._x1?this._append`M${c},${f}`:(Math.abs(this._x1-c)>Ra||Math.abs(this._y1-f)>Ra)&&this._append`L${c},${f}`,e&&(l<0&&(l=l%Da+Da),l>Fa?this._append`A${e},${e},0,1,${s},${t-a},${n-u}A${e},${e},0,1,${s},${this._x1=c},${this._y1=f}`:l>Ra&&this._append`A${e},${e},0,${+(l>=$a)},${s},${this._x1=t+e*Math.cos(i)},${this._y1=n+e*Math.sin(i)}`)}rect(t,n,e,r){this._append`M${this._x0=this._x1=+t},${this._y0=this._y1=+n}h${e=+e}v${+r}h${-e}Z`}toString(){return this._}};function Ia(){return new Ua}Ia.prototype=Ua.prototype;var Oa=Array.prototype.slice;function Ba(t){return function(){return t}}function Ya(t){return t.source}function La(t){return t.target}function ja(t){return t.radius}function Ha(t){return t.startAngle}function Xa(t){return t.endAngle}function Ga(){return 0}function Va(){return 10}function Wa(t){var n=Ya,e=La,r=ja,i=ja,o=Ha,a=Xa,u=Ga,c=null;function f(){var f,s=n.apply(this,arguments),l=e.apply(this,arguments),h=u.apply(this,arguments)/2,d=Oa.call(arguments),p=+r.apply(this,(d[0]=s,d)),g=o.apply(this,d)-Ea,y=a.apply(this,d)-Ea,v=+i.apply(this,(d[0]=l,d)),_=o.apply(this,d)-Ea,b=a.apply(this,d)-Ea;if(c||(c=f=Ia()),h>Ca&&(Ma(y-g)>2*h+Ca?y>g?(g+=h,y-=h):(g-=h,y+=h):g=y=(g+y)/2,Ma(b-_)>2*h+Ca?b>_?(_+=h,b-=h):(_-=h,b+=h):_=b=(_+b)/2),c.moveTo(p*Ta(g),p*Aa(g)),c.arc(0,0,p,g,y),g!==_||y!==b)if(t){var m=v-+t.apply(this,arguments),x=(_+b)/2;c.quadraticCurveTo(0,0,m*Ta(_),m*Aa(_)),c.lineTo(v*Ta(x),v*Aa(x)),c.lineTo(m*Ta(b),m*Aa(b))}else c.quadraticCurveTo(0,0,v*Ta(_),v*Aa(_)),c.arc(0,0,v,_,b);if(c.quadraticCurveTo(0,0,p*Ta(g),p*Aa(g)),c.closePath(),f)return c=null,f+""||null}return t&&(f.headRadius=function(n){return arguments.length?(t="function"==typeof n?n:Ba(+n),f):t}),f.radius=function(t){return arguments.length?(r=i="function"==typeof t?t:Ba(+t),f):r},f.sourceRadius=function(t){return arguments.length?(r="function"==typeof t?t:Ba(+t),f):r},f.targetRadius=function(t){return arguments.length?(i="function"==typeof t?t:Ba(+t),f):i},f.startAngle=function(t){return arguments.length?(o="function"==typeof t?t:Ba(+t),f):o},f.endAngle=function(t){return arguments.length?(a="function"==typeof t?t:Ba(+t),f):a},f.padAngle=function(t){return arguments.length?(u="function"==typeof t?t:Ba(+t),f):u},f.source=function(t){return arguments.length?(n=t,f):n},f.target=function(t){return arguments.length?(e=t,f):e},f.context=function(t){return arguments.length?(c=null==t?null:t,f):c},f}var Za=Array.prototype.slice;function Ka(t,n){return t-n}var Qa=t=>()=>t;function Ja(t,n){for(var e,r=-1,i=n.length;++rr!=d>r&&e<(h-f)*(r-s)/(d-s)+f&&(i=-i)}return i}function nu(t,n,e){var r,i,o,a;return function(t,n,e){return(n[0]-t[0])*(e[1]-t[1])==(e[0]-t[0])*(n[1]-t[1])}(t,n,e)&&(i=t[r=+(t[0]===n[0])],o=e[r],a=n[r],i<=o&&o<=a||a<=o&&o<=i)}function eu(){}var ru=[[],[[[1,1.5],[.5,1]]],[[[1.5,1],[1,1.5]]],[[[1.5,1],[.5,1]]],[[[1,.5],[1.5,1]]],[[[1,1.5],[.5,1]],[[1,.5],[1.5,1]]],[[[1,.5],[1,1.5]]],[[[1,.5],[.5,1]]],[[[.5,1],[1,.5]]],[[[1,1.5],[1,.5]]],[[[.5,1],[1,.5]],[[1.5,1],[1,1.5]]],[[[1.5,1],[1,.5]]],[[[.5,1],[1.5,1]]],[[[1,1.5],[1.5,1]]],[[[.5,1],[1,1.5]]],[]];function iu(){var t=1,n=1,e=K,r=u;function i(t){var n=e(t);if(Array.isArray(n))n=n.slice().sort(Ka);else{const e=M(t,ou);for(n=G(...Z(e[0],e[1],n),n);n[n.length-1]>=e[1];)n.pop();for(;n[1]o(t,n)))}function o(e,i){const o=null==i?NaN:+i;if(isNaN(o))throw new Error(`invalid value: ${i}`);var u=[],c=[];return function(e,r,i){var o,u,c,f,s,l,h=new Array,d=new Array;o=u=-1,f=au(e[0],r),ru[f<<1].forEach(p);for(;++o=r,ru[s<<2].forEach(p);for(;++o0?u.push([t]):c.push(t)})),c.forEach((function(t){for(var n,e=0,r=u.length;e0&&o0&&a=0&&o>=0))throw new Error("invalid size");return t=r,n=o,i},i.thresholds=function(t){return arguments.length?(e="function"==typeof t?t:Array.isArray(t)?Qa(Za.call(t)):Qa(t),i):e},i.smooth=function(t){return arguments.length?(r=t?u:eu,i):r===u},i}function ou(t){return isFinite(t)?t:NaN}function au(t,n){return null!=t&&+t>=n}function uu(t){return null==t||isNaN(t=+t)?-1/0:t}function cu(t,n,e,r){const i=r-n,o=e-n,a=isFinite(i)||isFinite(o)?i/o:Math.sign(i)/Math.sign(o);return isNaN(a)?t:t+a-.5}function fu(t){return t[0]}function su(t){return t[1]}function lu(){return 1}const hu=134217729,du=33306690738754706e-32;function pu(t,n,e,r,i){let o,a,u,c,f=n[0],s=r[0],l=0,h=0;s>f==s>-f?(o=f,f=n[++l]):(o=s,s=r[++h]);let d=0;if(lf==s>-f?(a=f+o,u=o-(a-f),f=n[++l]):(a=s+o,u=o-(a-s),s=r[++h]),o=a,0!==u&&(i[d++]=u);lf==s>-f?(a=o+f,c=a-o,u=o-(a-c)+(f-c),f=n[++l]):(a=o+s,c=a-o,u=o-(a-c)+(s-c),s=r[++h]),o=a,0!==u&&(i[d++]=u);for(;l=33306690738754716e-32*f?c:-function(t,n,e,r,i,o,a){let u,c,f,s,l,h,d,p,g,y,v,_,b,m,x,w,M,T;const A=t-i,S=e-i,E=n-o,N=r-o;m=A*N,h=hu*A,d=h-(h-A),p=A-d,h=hu*N,g=h-(h-N),y=N-g,x=p*y-(m-d*g-p*g-d*y),w=E*S,h=hu*E,d=h-(h-E),p=E-d,h=hu*S,g=h-(h-S),y=S-g,M=p*y-(w-d*g-p*g-d*y),v=x-M,l=x-v,_u[0]=x-(v+l)+(l-M),_=m+v,l=_-m,b=m-(_-l)+(v-l),v=b-w,l=b-v,_u[1]=b-(v+l)+(l-w),T=_+v,l=T-_,_u[2]=_-(T-l)+(v-l),_u[3]=T;let k=function(t,n){let e=n[0];for(let r=1;r=C||-k>=C)return k;if(l=t-A,u=t-(A+l)+(l-i),l=e-S,f=e-(S+l)+(l-i),l=n-E,c=n-(E+l)+(l-o),l=r-N,s=r-(N+l)+(l-o),0===u&&0===c&&0===f&&0===s)return k;if(C=vu*a+du*Math.abs(k),k+=A*s+N*u-(E*f+S*c),k>=C||-k>=C)return k;m=u*N,h=hu*u,d=h-(h-u),p=u-d,h=hu*N,g=h-(h-N),y=N-g,x=p*y-(m-d*g-p*g-d*y),w=c*S,h=hu*c,d=h-(h-c),p=c-d,h=hu*S,g=h-(h-S),y=S-g,M=p*y-(w-d*g-p*g-d*y),v=x-M,l=x-v,wu[0]=x-(v+l)+(l-M),_=m+v,l=_-m,b=m-(_-l)+(v-l),v=b-w,l=b-v,wu[1]=b-(v+l)+(l-w),T=_+v,l=T-_,wu[2]=_-(T-l)+(v-l),wu[3]=T;const P=pu(4,_u,4,wu,bu);m=A*s,h=hu*A,d=h-(h-A),p=A-d,h=hu*s,g=h-(h-s),y=s-g,x=p*y-(m-d*g-p*g-d*y),w=E*f,h=hu*E,d=h-(h-E),p=E-d,h=hu*f,g=h-(h-f),y=f-g,M=p*y-(w-d*g-p*g-d*y),v=x-M,l=x-v,wu[0]=x-(v+l)+(l-M),_=m+v,l=_-m,b=m-(_-l)+(v-l),v=b-w,l=b-v,wu[1]=b-(v+l)+(l-w),T=_+v,l=T-_,wu[2]=_-(T-l)+(v-l),wu[3]=T;const z=pu(P,bu,4,wu,mu);m=u*s,h=hu*u,d=h-(h-u),p=u-d,h=hu*s,g=h-(h-s),y=s-g,x=p*y-(m-d*g-p*g-d*y),w=c*f,h=hu*c,d=h-(h-c),p=c-d,h=hu*f,g=h-(h-f),y=f-g,M=p*y-(w-d*g-p*g-d*y),v=x-M,l=x-v,wu[0]=x-(v+l)+(l-M),_=m+v,l=_-m,b=m-(_-l)+(v-l),v=b-w,l=b-v,wu[1]=b-(v+l)+(l-w),T=_+v,l=T-_,wu[2]=_-(T-l)+(v-l),wu[3]=T;const $=pu(z,mu,4,wu,xu);return xu[$-1]}(t,n,e,r,i,o,f)}const Tu=Math.pow(2,-52),Au=new Uint32Array(512);class Su{static from(t,n=zu,e=$u){const r=t.length,i=new Float64Array(2*r);for(let o=0;o>1;if(n>0&&"number"!=typeof t[0])throw new Error("Expected coords to contain numbers.");this.coords=t;const e=Math.max(2*n-5,0);this._triangles=new Uint32Array(3*e),this._halfedges=new Int32Array(3*e),this._hashSize=Math.ceil(Math.sqrt(n)),this._hullPrev=new Uint32Array(n),this._hullNext=new Uint32Array(n),this._hullTri=new Uint32Array(n),this._hullHash=new Int32Array(this._hashSize),this._ids=new Uint32Array(n),this._dists=new Float64Array(n),this.update()}update(){const{coords:t,_hullPrev:n,_hullNext:e,_hullTri:r,_hullHash:i}=this,o=t.length>>1;let a=1/0,u=1/0,c=-1/0,f=-1/0;for(let n=0;nc&&(c=e),r>f&&(f=r),this._ids[n]=n}const s=(a+c)/2,l=(u+f)/2;let h,d,p;for(let n=0,e=1/0;n0&&(d=n,e=r)}let v=t[2*d],_=t[2*d+1],b=1/0;for(let n=0;nr&&(n[e++]=i,r=o)}return this.hull=n.subarray(0,e),this.triangles=new Uint32Array(0),void(this.halfedges=new Uint32Array(0))}if(Mu(g,y,v,_,m,x)<0){const t=d,n=v,e=_;d=p,v=m,_=x,p=t,m=n,x=e}const w=function(t,n,e,r,i,o){const a=e-t,u=r-n,c=i-t,f=o-n,s=a*a+u*u,l=c*c+f*f,h=.5/(a*f-u*c),d=t+(f*s-u*l)*h,p=n+(a*l-c*s)*h;return{x:d,y:p}}(g,y,v,_,m,x);this._cx=w.x,this._cy=w.y;for(let n=0;n0&&Math.abs(f-o)<=Tu&&Math.abs(s-a)<=Tu)continue;if(o=f,a=s,c===h||c===d||c===p)continue;let l=0;for(let t=0,n=this._hashKey(f,s);t=0;)if(y=g,y===l){y=-1;break}if(-1===y)continue;let v=this._addTriangle(y,c,e[y],-1,-1,r[y]);r[c]=this._legalize(v+2),r[y]=v,M++;let _=e[y];for(;g=e[_],Mu(f,s,t[2*_],t[2*_+1],t[2*g],t[2*g+1])<0;)v=this._addTriangle(_,c,g,r[c],-1,r[_]),r[c]=this._legalize(v+2),e[_]=_,M--,_=g;if(y===l)for(;g=n[y],Mu(f,s,t[2*g],t[2*g+1],t[2*y],t[2*y+1])<0;)v=this._addTriangle(g,c,y,-1,r[y],r[g]),this._legalize(v+2),r[g]=v,e[y]=y,M--,y=g;this._hullStart=n[c]=y,e[y]=n[_]=c,e[c]=_,i[this._hashKey(f,s)]=c,i[this._hashKey(t[2*y],t[2*y+1])]=y}this.hull=new Uint32Array(M);for(let t=0,n=this._hullStart;t0?3-e:1+e)/4}(t-this._cx,n-this._cy)*this._hashSize)%this._hashSize}_legalize(t){const{_triangles:n,_halfedges:e,coords:r}=this;let i=0,o=0;for(;;){const a=e[t],u=t-t%3;if(o=u+(t+2)%3,-1===a){if(0===i)break;t=Au[--i];continue}const c=a-a%3,f=u+(t+1)%3,s=c+(a+2)%3,l=n[o],h=n[t],d=n[f],p=n[s];if(Nu(r[2*l],r[2*l+1],r[2*h],r[2*h+1],r[2*d],r[2*d+1],r[2*p],r[2*p+1])){n[t]=p,n[a]=l;const r=e[s];if(-1===r){let n=this._hullStart;do{if(this._hullTri[n]===s){this._hullTri[n]=t;break}n=this._hullPrev[n]}while(n!==this._hullStart)}this._link(t,r),this._link(a,e[o]),this._link(o,s);const u=c+(a+1)%3;i=e&&n[t[a]]>o;)t[a+1]=t[a--];t[a+1]=r}else{let i=e+1,o=r;Pu(t,e+r>>1,i),n[t[e]]>n[t[r]]&&Pu(t,e,r),n[t[i]]>n[t[r]]&&Pu(t,i,r),n[t[e]]>n[t[i]]&&Pu(t,e,i);const a=t[i],u=n[a];for(;;){do{i++}while(n[t[i]]u);if(o=o-e?(Cu(t,n,i,r),Cu(t,n,e,o-1)):(Cu(t,n,e,o-1),Cu(t,n,i,r))}}function Pu(t,n,e){const r=t[n];t[n]=t[e],t[e]=r}function zu(t){return t[0]}function $u(t){return t[1]}const Du=1e-6;class Ru{constructor(){this._x0=this._y0=this._x1=this._y1=null,this._=""}moveTo(t,n){this._+=`M${this._x0=this._x1=+t},${this._y0=this._y1=+n}`}closePath(){null!==this._x1&&(this._x1=this._x0,this._y1=this._y0,this._+="Z")}lineTo(t,n){this._+=`L${this._x1=+t},${this._y1=+n}`}arc(t,n,e){const r=(t=+t)+(e=+e),i=n=+n;if(e<0)throw new Error("negative radius");null===this._x1?this._+=`M${r},${i}`:(Math.abs(this._x1-r)>Du||Math.abs(this._y1-i)>Du)&&(this._+="L"+r+","+i),e&&(this._+=`A${e},${e},0,1,1,${t-e},${n}A${e},${e},0,1,1,${this._x1=r},${this._y1=i}`)}rect(t,n,e,r){this._+=`M${this._x0=this._x1=+t},${this._y0=this._y1=+n}h${+e}v${+r}h${-e}Z`}value(){return this._||null}}class Fu{constructor(){this._=[]}moveTo(t,n){this._.push([t,n])}closePath(){this._.push(this._[0].slice())}lineTo(t,n){this._.push([t,n])}value(){return this._.length?this._:null}}class qu{constructor(t,[n,e,r,i]=[0,0,960,500]){if(!((r=+r)>=(n=+n)&&(i=+i)>=(e=+e)))throw new Error("invalid bounds");this.delaunay=t,this._circumcenters=new Float64Array(2*t.points.length),this.vectors=new Float64Array(2*t.points.length),this.xmax=r,this.xmin=n,this.ymax=i,this.ymin=e,this._init()}update(){return this.delaunay.update(),this._init(),this}_init(){const{delaunay:{points:t,hull:n,triangles:e},vectors:r}=this;let i,o;const a=this.circumcenters=this._circumcenters.subarray(0,e.length/3*2);for(let r,u,c=0,f=0,s=e.length;c1;)i-=2;for(let t=2;t0){if(n>=this.ymax)return null;(i=(this.ymax-n)/r)0){if(t>=this.xmax)return null;(i=(this.xmax-t)/e)this.xmax?2:0)|(nthis.ymax?8:0)}_simplify(t){if(t&&t.length>4){for(let n=0;n2&&function(t){const{triangles:n,coords:e}=t;for(let t=0;t1e-10)return!1}return!0}(t)){this.collinear=Int32Array.from({length:n.length/2},((t,n)=>n)).sort(((t,e)=>n[2*t]-n[2*e]||n[2*t+1]-n[2*e+1]));const t=this.collinear[0],e=this.collinear[this.collinear.length-1],r=[n[2*t],n[2*t+1],n[2*e],n[2*e+1]],i=1e-8*Math.hypot(r[3]-r[1],r[2]-r[0]);for(let t=0,e=n.length/2;t0&&(this.triangles=new Int32Array(3).fill(-1),this.halfedges=new Int32Array(3).fill(-1),this.triangles[0]=r[0],o[r[0]]=1,2===r.length&&(o[r[1]]=0,this.triangles[1]=r[1],this.triangles[2]=r[1]))}voronoi(t){return new qu(this,t)}*neighbors(t){const{inedges:n,hull:e,_hullIndex:r,halfedges:i,triangles:o,collinear:a}=this;if(a){const n=a.indexOf(t);return n>0&&(yield a[n-1]),void(n=0&&i!==e&&i!==r;)e=i;return i}_step(t,n,e){const{inedges:r,hull:i,_hullIndex:o,halfedges:a,triangles:u,points:c}=this;if(-1===r[t]||!c.length)return(t+1)%(c.length>>1);let f=t,s=Iu(n-c[2*t],2)+Iu(e-c[2*t+1],2);const l=r[t];let h=l;do{let r=u[h];const l=Iu(n-c[2*r],2)+Iu(e-c[2*r+1],2);if(l9999?"+"+Ku(n,6):Ku(n,4))+"-"+Ku(t.getUTCMonth()+1,2)+"-"+Ku(t.getUTCDate(),2)+(o?"T"+Ku(e,2)+":"+Ku(r,2)+":"+Ku(i,2)+"."+Ku(o,3)+"Z":i?"T"+Ku(e,2)+":"+Ku(r,2)+":"+Ku(i,2)+"Z":r||e?"T"+Ku(e,2)+":"+Ku(r,2)+"Z":"")}function Ju(t){var n=new RegExp('["'+t+"\n\r]"),e=t.charCodeAt(0);function r(t,n){var r,i=[],o=t.length,a=0,u=0,c=o<=0,f=!1;function s(){if(c)return Hu;if(f)return f=!1,ju;var n,r,i=a;if(t.charCodeAt(i)===Xu){for(;a++=o?c=!0:(r=t.charCodeAt(a++))===Gu?f=!0:r===Vu&&(f=!0,t.charCodeAt(a)===Gu&&++a),t.slice(i+1,n-1).replace(/""/g,'"')}for(;amc(n,e).then((n=>(new DOMParser).parseFromString(n,t)))}var Sc=Ac("application/xml"),Ec=Ac("text/html"),Nc=Ac("image/svg+xml");function kc(t,n,e,r){if(isNaN(n)||isNaN(e))return t;var i,o,a,u,c,f,s,l,h,d=t._root,p={data:r},g=t._x0,y=t._y0,v=t._x1,_=t._y1;if(!d)return t._root=p,t;for(;d.length;)if((f=n>=(o=(g+v)/2))?g=o:v=o,(s=e>=(a=(y+_)/2))?y=a:_=a,i=d,!(d=d[l=s<<1|f]))return i[l]=p,t;if(u=+t._x.call(null,d.data),c=+t._y.call(null,d.data),n===u&&e===c)return p.next=d,i?i[l]=p:t._root=p,t;do{i=i?i[l]=new Array(4):t._root=new Array(4),(f=n>=(o=(g+v)/2))?g=o:v=o,(s=e>=(a=(y+_)/2))?y=a:_=a}while((l=s<<1|f)==(h=(c>=a)<<1|u>=o));return i[h]=d,i[l]=p,t}function Cc(t,n,e,r,i){this.node=t,this.x0=n,this.y0=e,this.x1=r,this.y1=i}function Pc(t){return t[0]}function zc(t){return t[1]}function $c(t,n,e){var r=new Dc(null==n?Pc:n,null==e?zc:e,NaN,NaN,NaN,NaN);return null==t?r:r.addAll(t)}function Dc(t,n,e,r,i,o){this._x=t,this._y=n,this._x0=e,this._y0=r,this._x1=i,this._y1=o,this._root=void 0}function Rc(t){for(var n={data:t.data},e=n;t=t.next;)e=e.next={data:t.data};return n}var Fc=$c.prototype=Dc.prototype;function qc(t){return function(){return t}}function Uc(t){return 1e-6*(t()-.5)}function Ic(t){return t.x+t.vx}function Oc(t){return t.y+t.vy}function Bc(t){return t.index}function Yc(t,n){var e=t.get(n);if(!e)throw new Error("node not found: "+n);return e}Fc.copy=function(){var t,n,e=new Dc(this._x,this._y,this._x0,this._y0,this._x1,this._y1),r=this._root;if(!r)return e;if(!r.length)return e._root=Rc(r),e;for(t=[{source:r,target:e._root=new Array(4)}];r=t.pop();)for(var i=0;i<4;++i)(n=r.source[i])&&(n.length?t.push({source:n,target:r.target[i]=new Array(4)}):r.target[i]=Rc(n));return e},Fc.add=function(t){const n=+this._x.call(null,t),e=+this._y.call(null,t);return kc(this.cover(n,e),n,e,t)},Fc.addAll=function(t){var n,e,r,i,o=t.length,a=new Array(o),u=new Array(o),c=1/0,f=1/0,s=-1/0,l=-1/0;for(e=0;es&&(s=r),il&&(l=i));if(c>s||f>l)return this;for(this.cover(c,f).cover(s,l),e=0;et||t>=i||r>n||n>=o;)switch(u=(nh||(o=c.y0)>d||(a=c.x1)=v)<<1|t>=y)&&(c=p[p.length-1],p[p.length-1]=p[p.length-1-f],p[p.length-1-f]=c)}else{var _=t-+this._x.call(null,g.data),b=n-+this._y.call(null,g.data),m=_*_+b*b;if(m=(u=(p+y)/2))?p=u:y=u,(s=a>=(c=(g+v)/2))?g=c:v=c,n=d,!(d=d[l=s<<1|f]))return this;if(!d.length)break;(n[l+1&3]||n[l+2&3]||n[l+3&3])&&(e=n,h=l)}for(;d.data!==t;)if(r=d,!(d=d.next))return this;return(i=d.next)&&delete d.next,r?(i?r.next=i:delete r.next,this):n?(i?n[l]=i:delete n[l],(d=n[0]||n[1]||n[2]||n[3])&&d===(n[3]||n[2]||n[1]||n[0])&&!d.length&&(e?e[h]=d:this._root=d),this):(this._root=i,this)},Fc.removeAll=function(t){for(var n=0,e=t.length;n1?r[0]+r.slice(2):r,+t.slice(e+1)]}function Zc(t){return(t=Wc(Math.abs(t)))?t[1]:NaN}var Kc,Qc=/^(?:(.)?([<>=^]))?([+\-( ])?([$#])?(0)?(\d+)?(,)?(\.\d+)?(~)?([a-z%])?$/i;function Jc(t){if(!(n=Qc.exec(t)))throw new Error("invalid format: "+t);var n;return new tf({fill:n[1],align:n[2],sign:n[3],symbol:n[4],zero:n[5],width:n[6],comma:n[7],precision:n[8]&&n[8].slice(1),trim:n[9],type:n[10]})}function tf(t){this.fill=void 0===t.fill?" ":t.fill+"",this.align=void 0===t.align?">":t.align+"",this.sign=void 0===t.sign?"-":t.sign+"",this.symbol=void 0===t.symbol?"":t.symbol+"",this.zero=!!t.zero,this.width=void 0===t.width?void 0:+t.width,this.comma=!!t.comma,this.precision=void 0===t.precision?void 0:+t.precision,this.trim=!!t.trim,this.type=void 0===t.type?"":t.type+""}function nf(t,n){var e=Wc(t,n);if(!e)return t+"";var r=e[0],i=e[1];return i<0?"0."+new Array(-i).join("0")+r:r.length>i+1?r.slice(0,i+1)+"."+r.slice(i+1):r+new Array(i-r.length+2).join("0")}Jc.prototype=tf.prototype,tf.prototype.toString=function(){return this.fill+this.align+this.sign+this.symbol+(this.zero?"0":"")+(void 0===this.width?"":Math.max(1,0|this.width))+(this.comma?",":"")+(void 0===this.precision?"":"."+Math.max(0,0|this.precision))+(this.trim?"~":"")+this.type};var ef={"%":(t,n)=>(100*t).toFixed(n),b:t=>Math.round(t).toString(2),c:t=>t+"",d:function(t){return Math.abs(t=Math.round(t))>=1e21?t.toLocaleString("en").replace(/,/g,""):t.toString(10)},e:(t,n)=>t.toExponential(n),f:(t,n)=>t.toFixed(n),g:(t,n)=>t.toPrecision(n),o:t=>Math.round(t).toString(8),p:(t,n)=>nf(100*t,n),r:nf,s:function(t,n){var e=Wc(t,n);if(!e)return t+"";var r=e[0],i=e[1],o=i-(Kc=3*Math.max(-8,Math.min(8,Math.floor(i/3))))+1,a=r.length;return o===a?r:o>a?r+new Array(o-a+1).join("0"):o>0?r.slice(0,o)+"."+r.slice(o):"0."+new Array(1-o).join("0")+Wc(t,Math.max(0,n+o-1))[0]},X:t=>Math.round(t).toString(16).toUpperCase(),x:t=>Math.round(t).toString(16)};function rf(t){return t}var of,af=Array.prototype.map,uf=["y","z","a","f","p","n","µ","m","","k","M","G","T","P","E","Z","Y"];function cf(t){var n,e,r=void 0===t.grouping||void 0===t.thousands?rf:(n=af.call(t.grouping,Number),e=t.thousands+"",function(t,r){for(var i=t.length,o=[],a=0,u=n[0],c=0;i>0&&u>0&&(c+u+1>r&&(u=Math.max(1,r-c)),o.push(t.substring(i-=u,i+u)),!((c+=u+1)>r));)u=n[a=(a+1)%n.length];return o.reverse().join(e)}),i=void 0===t.currency?"":t.currency[0]+"",o=void 0===t.currency?"":t.currency[1]+"",a=void 0===t.decimal?".":t.decimal+"",u=void 0===t.numerals?rf:function(t){return function(n){return n.replace(/[0-9]/g,(function(n){return t[+n]}))}}(af.call(t.numerals,String)),c=void 0===t.percent?"%":t.percent+"",f=void 0===t.minus?"−":t.minus+"",s=void 0===t.nan?"NaN":t.nan+"";function l(t){var n=(t=Jc(t)).fill,e=t.align,l=t.sign,h=t.symbol,d=t.zero,p=t.width,g=t.comma,y=t.precision,v=t.trim,_=t.type;"n"===_?(g=!0,_="g"):ef[_]||(void 0===y&&(y=12),v=!0,_="g"),(d||"0"===n&&"="===e)&&(d=!0,n="0",e="=");var b="$"===h?i:"#"===h&&/[boxX]/.test(_)?"0"+_.toLowerCase():"",m="$"===h?o:/[%p]/.test(_)?c:"",x=ef[_],w=/[defgprs%]/.test(_);function M(t){var i,o,c,h=b,M=m;if("c"===_)M=x(t)+M,t="";else{var T=(t=+t)<0||1/t<0;if(t=isNaN(t)?s:x(Math.abs(t),y),v&&(t=function(t){t:for(var n,e=t.length,r=1,i=-1;r0&&(i=0)}return i>0?t.slice(0,i)+t.slice(n+1):t}(t)),T&&0==+t&&"+"!==l&&(T=!1),h=(T?"("===l?l:f:"-"===l||"("===l?"":l)+h,M=("s"===_?uf[8+Kc/3]:"")+M+(T&&"("===l?")":""),w)for(i=-1,o=t.length;++i(c=t.charCodeAt(i))||c>57){M=(46===c?a+t.slice(i+1):t.slice(i))+M,t=t.slice(0,i);break}}g&&!d&&(t=r(t,1/0));var A=h.length+t.length+M.length,S=A>1)+h+t+M+S.slice(A);break;default:t=S+h+t+M}return u(t)}return y=void 0===y?6:/[gprs]/.test(_)?Math.max(1,Math.min(21,y)):Math.max(0,Math.min(20,y)),M.toString=function(){return t+""},M}return{format:l,formatPrefix:function(t,n){var e=l(((t=Jc(t)).type="f",t)),r=3*Math.max(-8,Math.min(8,Math.floor(Zc(n)/3))),i=Math.pow(10,-r),o=uf[8+r/3];return function(t){return e(i*t)+o}}}}function ff(n){return of=cf(n),t.format=of.format,t.formatPrefix=of.formatPrefix,of}function sf(t){return Math.max(0,-Zc(Math.abs(t)))}function lf(t,n){return Math.max(0,3*Math.max(-8,Math.min(8,Math.floor(Zc(n)/3)))-Zc(Math.abs(t)))}function hf(t,n){return t=Math.abs(t),n=Math.abs(n)-t,Math.max(0,Zc(n)-Zc(t))+1}t.format=void 0,t.formatPrefix=void 0,ff({thousands:",",grouping:[3],currency:["$",""]});var df=1e-6,pf=1e-12,gf=Math.PI,yf=gf/2,vf=gf/4,_f=2*gf,bf=180/gf,mf=gf/180,xf=Math.abs,wf=Math.atan,Mf=Math.atan2,Tf=Math.cos,Af=Math.ceil,Sf=Math.exp,Ef=Math.hypot,Nf=Math.log,kf=Math.pow,Cf=Math.sin,Pf=Math.sign||function(t){return t>0?1:t<0?-1:0},zf=Math.sqrt,$f=Math.tan;function Df(t){return t>1?0:t<-1?gf:Math.acos(t)}function Rf(t){return t>1?yf:t<-1?-yf:Math.asin(t)}function Ff(t){return(t=Cf(t/2))*t}function qf(){}function Uf(t,n){t&&Of.hasOwnProperty(t.type)&&Of[t.type](t,n)}var If={Feature:function(t,n){Uf(t.geometry,n)},FeatureCollection:function(t,n){for(var e=t.features,r=-1,i=e.length;++r=0?1:-1,i=r*e,o=Tf(n=(n*=mf)/2+vf),a=Cf(n),u=Vf*a,c=Gf*o+u*Tf(i),f=u*r*Cf(i);as.add(Mf(f,c)),Xf=t,Gf=o,Vf=a}function ds(t){return[Mf(t[1],t[0]),Rf(t[2])]}function ps(t){var n=t[0],e=t[1],r=Tf(e);return[r*Tf(n),r*Cf(n),Cf(e)]}function gs(t,n){return t[0]*n[0]+t[1]*n[1]+t[2]*n[2]}function ys(t,n){return[t[1]*n[2]-t[2]*n[1],t[2]*n[0]-t[0]*n[2],t[0]*n[1]-t[1]*n[0]]}function vs(t,n){t[0]+=n[0],t[1]+=n[1],t[2]+=n[2]}function _s(t,n){return[t[0]*n,t[1]*n,t[2]*n]}function bs(t){var n=zf(t[0]*t[0]+t[1]*t[1]+t[2]*t[2]);t[0]/=n,t[1]/=n,t[2]/=n}var ms,xs,ws,Ms,Ts,As,Ss,Es,Ns,ks,Cs,Ps,zs,$s,Ds,Rs,Fs={point:qs,lineStart:Is,lineEnd:Os,polygonStart:function(){Fs.point=Bs,Fs.lineStart=Ys,Fs.lineEnd=Ls,rs=new T,cs.polygonStart()},polygonEnd:function(){cs.polygonEnd(),Fs.point=qs,Fs.lineStart=Is,Fs.lineEnd=Os,as<0?(Wf=-(Kf=180),Zf=-(Qf=90)):rs>df?Qf=90:rs<-df&&(Zf=-90),os[0]=Wf,os[1]=Kf},sphere:function(){Wf=-(Kf=180),Zf=-(Qf=90)}};function qs(t,n){is.push(os=[Wf=t,Kf=t]),nQf&&(Qf=n)}function Us(t,n){var e=ps([t*mf,n*mf]);if(es){var r=ys(es,e),i=ys([r[1],-r[0],0],r);bs(i),i=ds(i);var o,a=t-Jf,u=a>0?1:-1,c=i[0]*bf*u,f=xf(a)>180;f^(u*JfQf&&(Qf=o):f^(u*Jf<(c=(c+360)%360-180)&&cQf&&(Qf=n)),f?tjs(Wf,Kf)&&(Kf=t):js(t,Kf)>js(Wf,Kf)&&(Wf=t):Kf>=Wf?(tKf&&(Kf=t)):t>Jf?js(Wf,t)>js(Wf,Kf)&&(Kf=t):js(t,Kf)>js(Wf,Kf)&&(Wf=t)}else is.push(os=[Wf=t,Kf=t]);nQf&&(Qf=n),es=e,Jf=t}function Is(){Fs.point=Us}function Os(){os[0]=Wf,os[1]=Kf,Fs.point=qs,es=null}function Bs(t,n){if(es){var e=t-Jf;rs.add(xf(e)>180?e+(e>0?360:-360):e)}else ts=t,ns=n;cs.point(t,n),Us(t,n)}function Ys(){cs.lineStart()}function Ls(){Bs(ts,ns),cs.lineEnd(),xf(rs)>df&&(Wf=-(Kf=180)),os[0]=Wf,os[1]=Kf,es=null}function js(t,n){return(n-=t)<0?n+360:n}function Hs(t,n){return t[0]-n[0]}function Xs(t,n){return t[0]<=t[1]?t[0]<=n&&n<=t[1]:ngf&&(t-=Math.round(t/_f)*_f),[t,n]}function ul(t,n,e){return(t%=_f)?n||e?ol(fl(t),sl(n,e)):fl(t):n||e?sl(n,e):al}function cl(t){return function(n,e){return xf(n+=t)>gf&&(n-=Math.round(n/_f)*_f),[n,e]}}function fl(t){var n=cl(t);return n.invert=cl(-t),n}function sl(t,n){var e=Tf(t),r=Cf(t),i=Tf(n),o=Cf(n);function a(t,n){var a=Tf(n),u=Tf(t)*a,c=Cf(t)*a,f=Cf(n),s=f*e+u*r;return[Mf(c*i-s*o,u*e-f*r),Rf(s*i+c*o)]}return a.invert=function(t,n){var a=Tf(n),u=Tf(t)*a,c=Cf(t)*a,f=Cf(n),s=f*i-c*o;return[Mf(c*i+f*o,u*e+s*r),Rf(s*e-u*r)]},a}function ll(t){function n(n){return(n=t(n[0]*mf,n[1]*mf))[0]*=bf,n[1]*=bf,n}return t=ul(t[0]*mf,t[1]*mf,t.length>2?t[2]*mf:0),n.invert=function(n){return(n=t.invert(n[0]*mf,n[1]*mf))[0]*=bf,n[1]*=bf,n},n}function hl(t,n,e,r,i,o){if(e){var a=Tf(n),u=Cf(n),c=r*e;null==i?(i=n+r*_f,o=n-c/2):(i=dl(a,i),o=dl(a,o),(r>0?io)&&(i+=r*_f));for(var f,s=i;r>0?s>o:s1&&n.push(n.pop().concat(n.shift()))},result:function(){var e=n;return n=[],t=null,e}}}function gl(t,n){return xf(t[0]-n[0])=0;--o)i.point((s=f[o])[0],s[1]);else r(h.x,h.p.x,-1,i);h=h.p}f=(h=h.o).z,d=!d}while(!h.v);i.lineEnd()}}}function _l(t){if(n=t.length){for(var n,e,r=0,i=t[0];++r=0?1:-1,E=S*A,N=E>gf,k=y*w;if(c.add(Mf(k*S*Cf(E),v*M+k*Tf(E))),a+=N?A+S*_f:A,N^p>=e^m>=e){var C=ys(ps(d),ps(b));bs(C);var P=ys(o,C);bs(P);var z=(N^A>=0?-1:1)*Rf(P[2]);(r>z||r===z&&(C[0]||C[1]))&&(u+=N^A>=0?1:-1)}}return(a<-df||a0){for(l||(i.polygonStart(),l=!0),i.lineStart(),t=0;t1&&2&c&&h.push(h.pop().concat(h.shift())),a.push(h.filter(wl))}return h}}function wl(t){return t.length>1}function Ml(t,n){return((t=t.x)[0]<0?t[1]-yf-df:yf-t[1])-((n=n.x)[0]<0?n[1]-yf-df:yf-n[1])}al.invert=al;var Tl=xl((function(){return!0}),(function(t){var n,e=NaN,r=NaN,i=NaN;return{lineStart:function(){t.lineStart(),n=1},point:function(o,a){var u=o>0?gf:-gf,c=xf(o-e);xf(c-gf)0?yf:-yf),t.point(i,r),t.lineEnd(),t.lineStart(),t.point(u,r),t.point(o,r),n=0):i!==u&&c>=gf&&(xf(e-i)df?wf((Cf(n)*(o=Tf(r))*Cf(e)-Cf(r)*(i=Tf(n))*Cf(t))/(i*o*a)):(n+r)/2}(e,r,o,a),t.point(i,r),t.lineEnd(),t.lineStart(),t.point(u,r),n=0),t.point(e=o,r=a),i=u},lineEnd:function(){t.lineEnd(),e=r=NaN},clean:function(){return 2-n}}}),(function(t,n,e,r){var i;if(null==t)i=e*yf,r.point(-gf,i),r.point(0,i),r.point(gf,i),r.point(gf,0),r.point(gf,-i),r.point(0,-i),r.point(-gf,-i),r.point(-gf,0),r.point(-gf,i);else if(xf(t[0]-n[0])>df){var o=t[0]0,i=xf(n)>df;function o(t,e){return Tf(t)*Tf(e)>n}function a(t,e,r){var i=[1,0,0],o=ys(ps(t),ps(e)),a=gs(o,o),u=o[0],c=a-u*u;if(!c)return!r&&t;var f=n*a/c,s=-n*u/c,l=ys(i,o),h=_s(i,f);vs(h,_s(o,s));var d=l,p=gs(h,d),g=gs(d,d),y=p*p-g*(gs(h,h)-1);if(!(y<0)){var v=zf(y),_=_s(d,(-p-v)/g);if(vs(_,h),_=ds(_),!r)return _;var b,m=t[0],x=e[0],w=t[1],M=e[1];x0^_[1]<(xf(_[0]-m)gf^(m<=_[0]&&_[0]<=x)){var S=_s(d,(-p+v)/g);return vs(S,h),[_,ds(S)]}}}function u(n,e){var i=r?t:gf-t,o=0;return n<-i?o|=1:n>i&&(o|=2),e<-i?o|=4:e>i&&(o|=8),o}return xl(o,(function(t){var n,e,c,f,s;return{lineStart:function(){f=c=!1,s=1},point:function(l,h){var d,p=[l,h],g=o(l,h),y=r?g?0:u(l,h):g?u(l+(l<0?gf:-gf),h):0;if(!n&&(f=c=g)&&t.lineStart(),g!==c&&(!(d=a(n,p))||gl(n,d)||gl(p,d))&&(p[2]=1),g!==c)s=0,g?(t.lineStart(),d=a(p,n),t.point(d[0],d[1])):(d=a(n,p),t.point(d[0],d[1],2),t.lineEnd()),n=d;else if(i&&n&&r^g){var v;y&e||!(v=a(p,n,!0))||(s=0,r?(t.lineStart(),t.point(v[0][0],v[0][1]),t.point(v[1][0],v[1][1]),t.lineEnd()):(t.point(v[1][0],v[1][1]),t.lineEnd(),t.lineStart(),t.point(v[0][0],v[0][1],3)))}!g||n&&gl(n,p)||t.point(p[0],p[1]),n=p,c=g,e=y},lineEnd:function(){c&&t.lineEnd(),n=null},clean:function(){return s|(f&&c)<<1}}}),(function(n,r,i,o){hl(o,t,e,i,n,r)}),r?[0,-t]:[-gf,t-gf])}var Sl,El,Nl,kl,Cl=1e9,Pl=-Cl;function zl(t,n,e,r){function i(i,o){return t<=i&&i<=e&&n<=o&&o<=r}function o(i,o,u,f){var s=0,l=0;if(null==i||(s=a(i,u))!==(l=a(o,u))||c(i,o)<0^u>0)do{f.point(0===s||3===s?t:e,s>1?r:n)}while((s=(s+u+4)%4)!==l);else f.point(o[0],o[1])}function a(r,i){return xf(r[0]-t)0?0:3:xf(r[0]-e)0?2:1:xf(r[1]-n)0?1:0:i>0?3:2}function u(t,n){return c(t.x,n.x)}function c(t,n){var e=a(t,1),r=a(n,1);return e!==r?e-r:0===e?n[1]-t[1]:1===e?t[0]-n[0]:2===e?t[1]-n[1]:n[0]-t[0]}return function(a){var c,f,s,l,h,d,p,g,y,v,_,b=a,m=pl(),x={point:w,lineStart:function(){x.point=M,f&&f.push(s=[]);v=!0,y=!1,p=g=NaN},lineEnd:function(){c&&(M(l,h),d&&y&&m.rejoin(),c.push(m.result()));x.point=w,y&&b.lineEnd()},polygonStart:function(){b=m,c=[],f=[],_=!0},polygonEnd:function(){var n=function(){for(var n=0,e=0,i=f.length;er&&(h-o)*(r-a)>(d-a)*(t-o)&&++n:d<=r&&(h-o)*(r-a)<(d-a)*(t-o)&&--n;return n}(),e=_&&n,i=(c=ft(c)).length;(e||i)&&(a.polygonStart(),e&&(a.lineStart(),o(null,null,1,a),a.lineEnd()),i&&vl(c,u,n,o,a),a.polygonEnd());b=a,c=f=s=null}};function w(t,n){i(t,n)&&b.point(t,n)}function M(o,a){var u=i(o,a);if(f&&s.push([o,a]),v)l=o,h=a,d=u,v=!1,u&&(b.lineStart(),b.point(o,a));else if(u&&y)b.point(o,a);else{var c=[p=Math.max(Pl,Math.min(Cl,p)),g=Math.max(Pl,Math.min(Cl,g))],m=[o=Math.max(Pl,Math.min(Cl,o)),a=Math.max(Pl,Math.min(Cl,a))];!function(t,n,e,r,i,o){var a,u=t[0],c=t[1],f=0,s=1,l=n[0]-u,h=n[1]-c;if(a=e-u,l||!(a>0)){if(a/=l,l<0){if(a0){if(a>s)return;a>f&&(f=a)}if(a=i-u,l||!(a<0)){if(a/=l,l<0){if(a>s)return;a>f&&(f=a)}else if(l>0){if(a0)){if(a/=h,h<0){if(a0){if(a>s)return;a>f&&(f=a)}if(a=o-c,h||!(a<0)){if(a/=h,h<0){if(a>s)return;a>f&&(f=a)}else if(h>0){if(a0&&(t[0]=u+f*l,t[1]=c+f*h),s<1&&(n[0]=u+s*l,n[1]=c+s*h),!0}}}}}(c,m,t,n,e,r)?u&&(b.lineStart(),b.point(o,a),_=!1):(y||(b.lineStart(),b.point(c[0],c[1])),b.point(m[0],m[1]),u||b.lineEnd(),_=!1)}p=o,g=a,y=u}return x}}var $l={sphere:qf,point:qf,lineStart:function(){$l.point=Rl,$l.lineEnd=Dl},lineEnd:qf,polygonStart:qf,polygonEnd:qf};function Dl(){$l.point=$l.lineEnd=qf}function Rl(t,n){El=t*=mf,Nl=Cf(n*=mf),kl=Tf(n),$l.point=Fl}function Fl(t,n){t*=mf;var e=Cf(n*=mf),r=Tf(n),i=xf(t-El),o=Tf(i),a=r*Cf(i),u=kl*e-Nl*r*o,c=Nl*e+kl*r*o;Sl.add(Mf(zf(a*a+u*u),c)),El=t,Nl=e,kl=r}function ql(t){return Sl=new T,Lf(t,$l),+Sl}var Ul=[null,null],Il={type:"LineString",coordinates:Ul};function Ol(t,n){return Ul[0]=t,Ul[1]=n,ql(Il)}var Bl={Feature:function(t,n){return Ll(t.geometry,n)},FeatureCollection:function(t,n){for(var e=t.features,r=-1,i=e.length;++r0&&(i=Ol(t[o],t[o-1]))>0&&e<=i&&r<=i&&(e+r-i)*(1-Math.pow((e-r)/i,2))df})).map(c)).concat(lt(Af(o/d)*d,i,d).filter((function(t){return xf(t%g)>df})).map(f))}return v.lines=function(){return _().map((function(t){return{type:"LineString",coordinates:t}}))},v.outline=function(){return{type:"Polygon",coordinates:[s(r).concat(l(a).slice(1),s(e).reverse().slice(1),l(u).reverse().slice(1))]}},v.extent=function(t){return arguments.length?v.extentMajor(t).extentMinor(t):v.extentMinor()},v.extentMajor=function(t){return arguments.length?(r=+t[0][0],e=+t[1][0],u=+t[0][1],a=+t[1][1],r>e&&(t=r,r=e,e=t),u>a&&(t=u,u=a,a=t),v.precision(y)):[[r,u],[e,a]]},v.extentMinor=function(e){return arguments.length?(n=+e[0][0],t=+e[1][0],o=+e[0][1],i=+e[1][1],n>t&&(e=n,n=t,t=e),o>i&&(e=o,o=i,i=e),v.precision(y)):[[n,o],[t,i]]},v.step=function(t){return arguments.length?v.stepMajor(t).stepMinor(t):v.stepMinor()},v.stepMajor=function(t){return arguments.length?(p=+t[0],g=+t[1],v):[p,g]},v.stepMinor=function(t){return arguments.length?(h=+t[0],d=+t[1],v):[h,d]},v.precision=function(h){return arguments.length?(y=+h,c=Wl(o,i,90),f=Zl(n,t,y),s=Wl(u,a,90),l=Zl(r,e,y),v):y},v.extentMajor([[-180,-90+df],[180,90-df]]).extentMinor([[-180,-80-df],[180,80+df]])}var Ql,Jl,th,nh,eh=t=>t,rh=new T,ih=new T,oh={point:qf,lineStart:qf,lineEnd:qf,polygonStart:function(){oh.lineStart=ah,oh.lineEnd=fh},polygonEnd:function(){oh.lineStart=oh.lineEnd=oh.point=qf,rh.add(xf(ih)),ih=new T},result:function(){var t=rh/2;return rh=new T,t}};function ah(){oh.point=uh}function uh(t,n){oh.point=ch,Ql=th=t,Jl=nh=n}function ch(t,n){ih.add(nh*t-th*n),th=t,nh=n}function fh(){ch(Ql,Jl)}var sh=oh,lh=1/0,hh=lh,dh=-lh,ph=dh,gh={point:function(t,n){tdh&&(dh=t);nph&&(ph=n)},lineStart:qf,lineEnd:qf,polygonStart:qf,polygonEnd:qf,result:function(){var t=[[lh,hh],[dh,ph]];return dh=ph=-(hh=lh=1/0),t}};var yh,vh,_h,bh,mh=gh,xh=0,wh=0,Mh=0,Th=0,Ah=0,Sh=0,Eh=0,Nh=0,kh=0,Ch={point:Ph,lineStart:zh,lineEnd:Rh,polygonStart:function(){Ch.lineStart=Fh,Ch.lineEnd=qh},polygonEnd:function(){Ch.point=Ph,Ch.lineStart=zh,Ch.lineEnd=Rh},result:function(){var t=kh?[Eh/kh,Nh/kh]:Sh?[Th/Sh,Ah/Sh]:Mh?[xh/Mh,wh/Mh]:[NaN,NaN];return xh=wh=Mh=Th=Ah=Sh=Eh=Nh=kh=0,t}};function Ph(t,n){xh+=t,wh+=n,++Mh}function zh(){Ch.point=$h}function $h(t,n){Ch.point=Dh,Ph(_h=t,bh=n)}function Dh(t,n){var e=t-_h,r=n-bh,i=zf(e*e+r*r);Th+=i*(_h+t)/2,Ah+=i*(bh+n)/2,Sh+=i,Ph(_h=t,bh=n)}function Rh(){Ch.point=Ph}function Fh(){Ch.point=Uh}function qh(){Ih(yh,vh)}function Uh(t,n){Ch.point=Ih,Ph(yh=_h=t,vh=bh=n)}function Ih(t,n){var e=t-_h,r=n-bh,i=zf(e*e+r*r);Th+=i*(_h+t)/2,Ah+=i*(bh+n)/2,Sh+=i,Eh+=(i=bh*t-_h*n)*(_h+t),Nh+=i*(bh+n),kh+=3*i,Ph(_h=t,bh=n)}var Oh=Ch;function Bh(t){this._context=t}Bh.prototype={_radius:4.5,pointRadius:function(t){return this._radius=t,this},polygonStart:function(){this._line=0},polygonEnd:function(){this._line=NaN},lineStart:function(){this._point=0},lineEnd:function(){0===this._line&&this._context.closePath(),this._point=NaN},point:function(t,n){switch(this._point){case 0:this._context.moveTo(t,n),this._point=1;break;case 1:this._context.lineTo(t,n);break;default:this._context.moveTo(t+this._radius,n),this._context.arc(t,n,this._radius,0,_f)}},result:qf};var Yh,Lh,jh,Hh,Xh,Gh=new T,Vh={point:qf,lineStart:function(){Vh.point=Wh},lineEnd:function(){Yh&&Zh(Lh,jh),Vh.point=qf},polygonStart:function(){Yh=!0},polygonEnd:function(){Yh=null},result:function(){var t=+Gh;return Gh=new T,t}};function Wh(t,n){Vh.point=Zh,Lh=Hh=t,jh=Xh=n}function Zh(t,n){Hh-=t,Xh-=n,Gh.add(zf(Hh*Hh+Xh*Xh)),Hh=t,Xh=n}var Kh=Vh;let Qh,Jh,td,nd;class ed{constructor(t){this._append=null==t?rd:function(t){const n=Math.floor(t);if(!(n>=0))throw new RangeError(`invalid digits: ${t}`);if(n>15)return rd;if(n!==Qh){const t=10**n;Qh=n,Jh=function(n){let e=1;this._+=n[0];for(const r=n.length;e4*n&&g--){var m=a+h,x=u+d,w=c+p,M=zf(m*m+x*x+w*w),T=Rf(w/=M),A=xf(xf(w)-1)n||xf((v*k+_*C)/b-.5)>.3||a*h+u*d+c*p2?t[2]%360*mf:0,k()):[y*bf,v*bf,_*bf]},E.angle=function(t){return arguments.length?(b=t%360*mf,k()):b*bf},E.reflectX=function(t){return arguments.length?(m=t?-1:1,k()):m<0},E.reflectY=function(t){return arguments.length?(x=t?-1:1,k()):x<0},E.precision=function(t){return arguments.length?(a=dd(u,S=t*t),C()):zf(S)},E.fitExtent=function(t,n){return ud(E,t,n)},E.fitSize=function(t,n){return cd(E,t,n)},E.fitWidth=function(t,n){return fd(E,t,n)},E.fitHeight=function(t,n){return sd(E,t,n)},function(){return n=t.apply(this,arguments),E.invert=n.invert&&N,k()}}function _d(t){var n=0,e=gf/3,r=vd(t),i=r(n,e);return i.parallels=function(t){return arguments.length?r(n=t[0]*mf,e=t[1]*mf):[n*bf,e*bf]},i}function bd(t,n){var e=Cf(t),r=(e+Cf(n))/2;if(xf(r)0?n<-yf+df&&(n=-yf+df):n>yf-df&&(n=yf-df);var e=i/kf(Nd(n),r);return[e*Cf(r*t),i-e*Tf(r*t)]}return o.invert=function(t,n){var e=i-n,o=Pf(r)*zf(t*t+e*e),a=Mf(t,xf(e))*Pf(e);return e*r<0&&(a-=gf*Pf(t)*Pf(e)),[a/r,2*wf(kf(i/o,1/r))-yf]},o}function Cd(t,n){return[t,n]}function Pd(t,n){var e=Tf(t),r=t===n?Cf(t):(e-Tf(n))/(n-t),i=e/r+t;if(xf(r)=0;)n+=e[r].value;else n=1;t.value=n}function Gd(t,n){t instanceof Map?(t=[void 0,t],void 0===n&&(n=Wd)):void 0===n&&(n=Vd);for(var e,r,i,o,a,u=new Qd(t),c=[u];e=c.pop();)if((i=n(e.data))&&(a=(i=Array.from(i)).length))for(e.children=i,o=a-1;o>=0;--o)c.push(r=i[o]=new Qd(i[o])),r.parent=e,r.depth=e.depth+1;return u.eachBefore(Kd)}function Vd(t){return t.children}function Wd(t){return Array.isArray(t)?t[1]:null}function Zd(t){void 0!==t.data.value&&(t.value=t.data.value),t.data=t.data.data}function Kd(t){var n=0;do{t.height=n}while((t=t.parent)&&t.height<++n)}function Qd(t){this.data=t,this.depth=this.height=0,this.parent=null}function Jd(t){return null==t?null:tp(t)}function tp(t){if("function"!=typeof t)throw new Error;return t}function np(){return 0}function ep(t){return function(){return t}}qd.invert=function(t,n){for(var e,r=n,i=r*r,o=i*i*i,a=0;a<12&&(o=(i=(r-=e=(r*(zd+$d*i+o*(Dd+Rd*i))-n)/(zd+3*$d*i+o*(7*Dd+9*Rd*i)))*r)*i*i,!(xf(e)df&&--i>0);return[t/(.8707+(o=r*r)*(o*(o*o*o*(.003971-.001529*o)-.013791)-.131979)),r]},Od.invert=Md(Rf),Bd.invert=Md((function(t){return 2*wf(t)})),Yd.invert=function(t,n){return[-n,2*wf(Sf(t))-yf]},Qd.prototype=Gd.prototype={constructor:Qd,count:function(){return this.eachAfter(Xd)},each:function(t,n){let e=-1;for(const r of this)t.call(n,r,++e,this);return this},eachAfter:function(t,n){for(var e,r,i,o=this,a=[o],u=[],c=-1;o=a.pop();)if(u.push(o),e=o.children)for(r=0,i=e.length;r=0;--r)o.push(e[r]);return this},find:function(t,n){let e=-1;for(const r of this)if(t.call(n,r,++e,this))return r},sum:function(t){return this.eachAfter((function(n){for(var e=+t(n.data)||0,r=n.children,i=r&&r.length;--i>=0;)e+=r[i].value;n.value=e}))},sort:function(t){return this.eachBefore((function(n){n.children&&n.children.sort(t)}))},path:function(t){for(var n=this,e=function(t,n){if(t===n)return t;var e=t.ancestors(),r=n.ancestors(),i=null;t=e.pop(),n=r.pop();for(;t===n;)i=t,t=e.pop(),n=r.pop();return i}(n,t),r=[n];n!==e;)n=n.parent,r.push(n);for(var i=r.length;t!==e;)r.splice(i,0,t),t=t.parent;return r},ancestors:function(){for(var t=this,n=[t];t=t.parent;)n.push(t);return n},descendants:function(){return Array.from(this)},leaves:function(){var t=[];return this.eachBefore((function(n){n.children||t.push(n)})),t},links:function(){var t=this,n=[];return t.each((function(e){e!==t&&n.push({source:e.parent,target:e})})),n},copy:function(){return Gd(this).eachBefore(Zd)},[Symbol.iterator]:function*(){var t,n,e,r,i=this,o=[i];do{for(t=o.reverse(),o=[];i=t.pop();)if(yield i,n=i.children)for(e=0,r=n.length;e(t=(rp*t+ip)%op)/op}function up(t,n){for(var e,r,i=0,o=(t=function(t,n){let e,r,i=t.length;for(;i;)r=n()*i--|0,e=t[i],t[i]=t[r],t[r]=e;return t}(Array.from(t),n)).length,a=[];i0&&e*e>r*r+i*i}function lp(t,n){for(var e=0;e1e-6?(E+Math.sqrt(E*E-4*S*N))/(2*S):N/E);return{x:r+w+M*k,y:i+T+A*k,r:k}}function gp(t,n,e){var r,i,o,a,u=t.x-n.x,c=t.y-n.y,f=u*u+c*c;f?(i=n.r+e.r,i*=i,a=t.r+e.r,i>(a*=a)?(r=(f+a-i)/(2*f),o=Math.sqrt(Math.max(0,a/f-r*r)),e.x=t.x-r*u-o*c,e.y=t.y-r*c+o*u):(r=(f+i-a)/(2*f),o=Math.sqrt(Math.max(0,i/f-r*r)),e.x=n.x+r*u-o*c,e.y=n.y+r*c+o*u)):(e.x=n.x+e.r,e.y=n.y)}function yp(t,n){var e=t.r+n.r-1e-6,r=n.x-t.x,i=n.y-t.y;return e>0&&e*e>r*r+i*i}function vp(t){var n=t._,e=t.next._,r=n.r+e.r,i=(n.x*e.r+e.x*n.r)/r,o=(n.y*e.r+e.y*n.r)/r;return i*i+o*o}function _p(t){this._=t,this.next=null,this.previous=null}function bp(t,n){if(!(o=(t=function(t){return"object"==typeof t&&"length"in t?t:Array.from(t)}(t)).length))return 0;var e,r,i,o,a,u,c,f,s,l,h;if((e=t[0]).x=0,e.y=0,!(o>1))return e.r;if(r=t[1],e.x=-r.r,r.x=e.r,r.y=0,!(o>2))return e.r+r.r;gp(r,e,i=t[2]),e=new _p(e),r=new _p(r),i=new _p(i),e.next=i.previous=r,r.next=e.previous=i,i.next=r.previous=e;t:for(c=3;c1&&!zp(t,n););return t.slice(0,n)}function zp(t,n){if("/"===t[n]){let e=0;for(;n>0&&"\\"===t[--n];)++e;if(!(1&e))return!0}return!1}function $p(t,n){return t.parent===n.parent?1:2}function Dp(t){var n=t.children;return n?n[0]:t.t}function Rp(t){var n=t.children;return n?n[n.length-1]:t.t}function Fp(t,n,e){var r=e/(n.i-t.i);n.c-=r,n.s+=e,t.c+=r,n.z+=e,n.m+=e}function qp(t,n,e){return t.a.parent===n.parent?t.a:e}function Up(t,n){this._=t,this.parent=null,this.children=null,this.A=null,this.a=this,this.z=0,this.m=0,this.c=0,this.s=0,this.t=null,this.i=n}function Ip(t,n,e,r,i){for(var o,a=t.children,u=-1,c=a.length,f=t.value&&(i-e)/t.value;++uh&&(h=u),y=s*s*g,(d=Math.max(h/y,y/l))>p){s-=u;break}p=d}v.push(a={value:s,dice:c1?n:1)},e}(Op);var Lp=function t(n){function e(t,e,r,i,o){if((a=t._squarify)&&a.ratio===n)for(var a,u,c,f,s,l=-1,h=a.length,d=t.value;++l1?n:1)},e}(Op);function jp(t,n,e){return(n[0]-t[0])*(e[1]-t[1])-(n[1]-t[1])*(e[0]-t[0])}function Hp(t,n){return t[0]-n[0]||t[1]-n[1]}function Xp(t){const n=t.length,e=[0,1];let r,i=2;for(r=2;r1&&jp(t[e[i-2]],t[e[i-1]],t[r])<=0;)--i;e[i++]=r}return e.slice(0,i)}var Gp=Math.random,Vp=function t(n){function e(t,e){return t=null==t?0:+t,e=null==e?1:+e,1===arguments.length?(e=t,t=0):e-=t,function(){return n()*e+t}}return e.source=t,e}(Gp),Wp=function t(n){function e(t,e){return arguments.length<2&&(e=t,t=0),t=Math.floor(t),e=Math.floor(e)-t,function(){return Math.floor(n()*e+t)}}return e.source=t,e}(Gp),Zp=function t(n){function e(t,e){var r,i;return t=null==t?0:+t,e=null==e?1:+e,function(){var o;if(null!=r)o=r,r=null;else do{r=2*n()-1,o=2*n()-1,i=r*r+o*o}while(!i||i>1);return t+e*o*Math.sqrt(-2*Math.log(i)/i)}}return e.source=t,e}(Gp),Kp=function t(n){var e=Zp.source(n);function r(){var t=e.apply(this,arguments);return function(){return Math.exp(t())}}return r.source=t,r}(Gp),Qp=function t(n){function e(t){return(t=+t)<=0?()=>0:function(){for(var e=0,r=t;r>1;--r)e+=n();return e+r*n()}}return e.source=t,e}(Gp),Jp=function t(n){var e=Qp.source(n);function r(t){if(0==(t=+t))return n;var r=e(t);return function(){return r()/t}}return r.source=t,r}(Gp),tg=function t(n){function e(t){return function(){return-Math.log1p(-n())/t}}return e.source=t,e}(Gp),ng=function t(n){function e(t){if((t=+t)<0)throw new RangeError("invalid alpha");return t=1/-t,function(){return Math.pow(1-n(),t)}}return e.source=t,e}(Gp),eg=function t(n){function e(t){if((t=+t)<0||t>1)throw new RangeError("invalid p");return function(){return Math.floor(n()+t)}}return e.source=t,e}(Gp),rg=function t(n){function e(t){if((t=+t)<0||t>1)throw new RangeError("invalid p");return 0===t?()=>1/0:1===t?()=>1:(t=Math.log1p(-t),function(){return 1+Math.floor(Math.log1p(-n())/t)})}return e.source=t,e}(Gp),ig=function t(n){var e=Zp.source(n)();function r(t,r){if((t=+t)<0)throw new RangeError("invalid k");if(0===t)return()=>0;if(r=null==r?1:+r,1===t)return()=>-Math.log1p(-n())*r;var i=(t<1?t+1:t)-1/3,o=1/(3*Math.sqrt(i)),a=t<1?()=>Math.pow(n(),1/t):()=>1;return function(){do{do{var t=e(),u=1+o*t}while(u<=0);u*=u*u;var c=1-n()}while(c>=1-.0331*t*t*t*t&&Math.log(c)>=.5*t*t+i*(1-u+Math.log(u)));return i*u*a()*r}}return r.source=t,r}(Gp),og=function t(n){var e=ig.source(n);function r(t,n){var r=e(t),i=e(n);return function(){var t=r();return 0===t?0:t/(t+i())}}return r.source=t,r}(Gp),ag=function t(n){var e=rg.source(n),r=og.source(n);function i(t,n){return t=+t,(n=+n)>=1?()=>t:n<=0?()=>0:function(){for(var i=0,o=t,a=n;o*a>16&&o*(1-a)>16;){var u=Math.floor((o+1)*a),c=r(u,o-u+1)();c<=a?(i+=u,o-=u,a=(a-c)/(1-c)):(o=u-1,a/=c)}for(var f=a<.5,s=e(f?a:1-a),l=s(),h=0;l<=o;++h)l+=s();return i+(f?h:o-h)}}return i.source=t,i}(Gp),ug=function t(n){function e(t,e,r){var i;return 0==(t=+t)?i=t=>-Math.log(t):(t=1/t,i=n=>Math.pow(n,t)),e=null==e?0:+e,r=null==r?1:+r,function(){return e+r*i(-Math.log1p(-n()))}}return e.source=t,e}(Gp),cg=function t(n){function e(t,e){return t=null==t?0:+t,e=null==e?1:+e,function(){return t+e*Math.tan(Math.PI*n())}}return e.source=t,e}(Gp),fg=function t(n){function e(t,e){return t=null==t?0:+t,e=null==e?1:+e,function(){var r=n();return t+e*Math.log(r/(1-r))}}return e.source=t,e}(Gp),sg=function t(n){var e=ig.source(n),r=ag.source(n);function i(t){return function(){for(var i=0,o=t;o>16;){var a=Math.floor(.875*o),u=e(a)();if(u>o)return i+r(a-1,o/u)();i+=a,o-=u}for(var c=-Math.log1p(-n()),f=0;c<=o;++f)c-=Math.log1p(-n());return i+f}}return i.source=t,i}(Gp);const lg=1/4294967296;function hg(t,n){switch(arguments.length){case 0:break;case 1:this.range(t);break;default:this.range(n).domain(t)}return this}function dg(t,n){switch(arguments.length){case 0:break;case 1:"function"==typeof t?this.interpolator(t):this.range(t);break;default:this.domain(t),"function"==typeof n?this.interpolator(n):this.range(n)}return this}const pg=Symbol("implicit");function gg(){var t=new InternMap,n=[],e=[],r=pg;function i(i){let o=t.get(i);if(void 0===o){if(r!==pg)return r;t.set(i,o=n.push(i)-1)}return e[o%e.length]}return i.domain=function(e){if(!arguments.length)return n.slice();n=[],t=new InternMap;for(const r of e)t.has(r)||t.set(r,n.push(r)-1);return i},i.range=function(t){return arguments.length?(e=Array.from(t),i):e.slice()},i.unknown=function(t){return arguments.length?(r=t,i):r},i.copy=function(){return gg(n,e).unknown(r)},hg.apply(i,arguments),i}function yg(){var t,n,e=gg().unknown(void 0),r=e.domain,i=e.range,o=0,a=1,u=!1,c=0,f=0,s=.5;function l(){var e=r().length,l=an&&(e=t,t=n,n=e),function(e){return Math.max(t,Math.min(n,e))}}(a[0],a[t-1])),r=t>2?Mg:wg,i=o=null,l}function l(n){return null==n||isNaN(n=+n)?e:(i||(i=r(a.map(t),u,c)))(t(f(n)))}return l.invert=function(e){return f(n((o||(o=r(u,a.map(t),Yr)))(e)))},l.domain=function(t){return arguments.length?(a=Array.from(t,_g),s()):a.slice()},l.range=function(t){return arguments.length?(u=Array.from(t),s()):u.slice()},l.rangeRound=function(t){return u=Array.from(t),c=Vr,s()},l.clamp=function(t){return arguments.length?(f=!!t||mg,s()):f!==mg},l.interpolate=function(t){return arguments.length?(c=t,s()):c},l.unknown=function(t){return arguments.length?(e=t,l):e},function(e,r){return t=e,n=r,s()}}function Sg(){return Ag()(mg,mg)}function Eg(n,e,r,i){var o,a=W(n,e,r);switch((i=Jc(null==i?",f":i)).type){case"s":var u=Math.max(Math.abs(n),Math.abs(e));return null!=i.precision||isNaN(o=lf(a,u))||(i.precision=o),t.formatPrefix(i,u);case"":case"e":case"g":case"p":case"r":null!=i.precision||isNaN(o=hf(a,Math.max(Math.abs(n),Math.abs(e))))||(i.precision=o-("e"===i.type));break;case"f":case"%":null!=i.precision||isNaN(o=sf(a))||(i.precision=o-2*("%"===i.type))}return t.format(i)}function Ng(t){var n=t.domain;return t.ticks=function(t){var e=n();return G(e[0],e[e.length-1],null==t?10:t)},t.tickFormat=function(t,e){var r=n();return Eg(r[0],r[r.length-1],null==t?10:t,e)},t.nice=function(e){null==e&&(e=10);var r,i,o=n(),a=0,u=o.length-1,c=o[a],f=o[u],s=10;for(f0;){if((i=V(c,f,e))===r)return o[a]=c,o[u]=f,n(o);if(i>0)c=Math.floor(c/i)*i,f=Math.ceil(f/i)*i;else{if(!(i<0))break;c=Math.ceil(c*i)/i,f=Math.floor(f*i)/i}r=i}return t},t}function kg(t,n){var e,r=0,i=(t=t.slice()).length-1,o=t[r],a=t[i];return a-t(-n,e)}function Fg(n){const e=n(Cg,Pg),r=e.domain;let i,o,a=10;function u(){return i=function(t){return t===Math.E?Math.log:10===t&&Math.log10||2===t&&Math.log2||(t=Math.log(t),n=>Math.log(n)/t)}(a),o=function(t){return 10===t?Dg:t===Math.E?Math.exp:n=>Math.pow(t,n)}(a),r()[0]<0?(i=Rg(i),o=Rg(o),n(zg,$g)):n(Cg,Pg),e}return e.base=function(t){return arguments.length?(a=+t,u()):a},e.domain=function(t){return arguments.length?(r(t),u()):r()},e.ticks=t=>{const n=r();let e=n[0],u=n[n.length-1];const c=u0){for(;l<=h;++l)for(f=1;fu)break;p.push(s)}}else for(;l<=h;++l)for(f=a-1;f>=1;--f)if(s=l>0?f/o(-l):f*o(l),!(su)break;p.push(s)}2*p.length{if(null==n&&(n=10),null==r&&(r=10===a?"s":","),"function"!=typeof r&&(a%1||null!=(r=Jc(r)).precision||(r.trim=!0),r=t.format(r)),n===1/0)return r;const u=Math.max(1,a*n/e.ticks().length);return t=>{let n=t/o(Math.round(i(t)));return n*ar(kg(r(),{floor:t=>o(Math.floor(i(t))),ceil:t=>o(Math.ceil(i(t)))})),e}function qg(t){return function(n){return Math.sign(n)*Math.log1p(Math.abs(n/t))}}function Ug(t){return function(n){return Math.sign(n)*Math.expm1(Math.abs(n))*t}}function Ig(t){var n=1,e=t(qg(n),Ug(n));return e.constant=function(e){return arguments.length?t(qg(n=+e),Ug(n)):n},Ng(e)}function Og(t){return function(n){return n<0?-Math.pow(-n,t):Math.pow(n,t)}}function Bg(t){return t<0?-Math.sqrt(-t):Math.sqrt(t)}function Yg(t){return t<0?-t*t:t*t}function Lg(t){var n=t(mg,mg),e=1;return n.exponent=function(n){return arguments.length?1===(e=+n)?t(mg,mg):.5===e?t(Bg,Yg):t(Og(e),Og(1/e)):e},Ng(n)}function jg(){var t=Lg(Ag());return t.copy=function(){return Tg(t,jg()).exponent(t.exponent())},hg.apply(t,arguments),t}function Hg(t){return Math.sign(t)*t*t}const Xg=new Date,Gg=new Date;function Vg(t,n,e,r){function i(n){return t(n=0===arguments.length?new Date:new Date(+n)),n}return i.floor=n=>(t(n=new Date(+n)),n),i.ceil=e=>(t(e=new Date(e-1)),n(e,1),t(e),e),i.round=t=>{const n=i(t),e=i.ceil(t);return t-n(n(t=new Date(+t),null==e?1:Math.floor(e)),t),i.range=(e,r,o)=>{const a=[];if(e=i.ceil(e),o=null==o?1:Math.floor(o),!(e0))return a;let u;do{a.push(u=new Date(+e)),n(e,o),t(e)}while(uVg((n=>{if(n>=n)for(;t(n),!e(n);)n.setTime(n-1)}),((t,r)=>{if(t>=t)if(r<0)for(;++r<=0;)for(;n(t,-1),!e(t););else for(;--r>=0;)for(;n(t,1),!e(t););})),e&&(i.count=(n,r)=>(Xg.setTime(+n),Gg.setTime(+r),t(Xg),t(Gg),Math.floor(e(Xg,Gg))),i.every=t=>(t=Math.floor(t),isFinite(t)&&t>0?t>1?i.filter(r?n=>r(n)%t==0:n=>i.count(0,n)%t==0):i:null)),i}const Wg=Vg((()=>{}),((t,n)=>{t.setTime(+t+n)}),((t,n)=>n-t));Wg.every=t=>(t=Math.floor(t),isFinite(t)&&t>0?t>1?Vg((n=>{n.setTime(Math.floor(n/t)*t)}),((n,e)=>{n.setTime(+n+e*t)}),((n,e)=>(e-n)/t)):Wg:null);const Zg=Wg.range,Kg=1e3,Qg=6e4,Jg=36e5,ty=864e5,ny=6048e5,ey=2592e6,ry=31536e6,iy=Vg((t=>{t.setTime(t-t.getMilliseconds())}),((t,n)=>{t.setTime(+t+n*Kg)}),((t,n)=>(n-t)/Kg),(t=>t.getUTCSeconds())),oy=iy.range,ay=Vg((t=>{t.setTime(t-t.getMilliseconds()-t.getSeconds()*Kg)}),((t,n)=>{t.setTime(+t+n*Qg)}),((t,n)=>(n-t)/Qg),(t=>t.getMinutes())),uy=ay.range,cy=Vg((t=>{t.setUTCSeconds(0,0)}),((t,n)=>{t.setTime(+t+n*Qg)}),((t,n)=>(n-t)/Qg),(t=>t.getUTCMinutes())),fy=cy.range,sy=Vg((t=>{t.setTime(t-t.getMilliseconds()-t.getSeconds()*Kg-t.getMinutes()*Qg)}),((t,n)=>{t.setTime(+t+n*Jg)}),((t,n)=>(n-t)/Jg),(t=>t.getHours())),ly=sy.range,hy=Vg((t=>{t.setUTCMinutes(0,0,0)}),((t,n)=>{t.setTime(+t+n*Jg)}),((t,n)=>(n-t)/Jg),(t=>t.getUTCHours())),dy=hy.range,py=Vg((t=>t.setHours(0,0,0,0)),((t,n)=>t.setDate(t.getDate()+n)),((t,n)=>(n-t-(n.getTimezoneOffset()-t.getTimezoneOffset())*Qg)/ty),(t=>t.getDate()-1)),gy=py.range,yy=Vg((t=>{t.setUTCHours(0,0,0,0)}),((t,n)=>{t.setUTCDate(t.getUTCDate()+n)}),((t,n)=>(n-t)/ty),(t=>t.getUTCDate()-1)),vy=yy.range,_y=Vg((t=>{t.setUTCHours(0,0,0,0)}),((t,n)=>{t.setUTCDate(t.getUTCDate()+n)}),((t,n)=>(n-t)/ty),(t=>Math.floor(t/ty))),by=_y.range;function my(t){return Vg((n=>{n.setDate(n.getDate()-(n.getDay()+7-t)%7),n.setHours(0,0,0,0)}),((t,n)=>{t.setDate(t.getDate()+7*n)}),((t,n)=>(n-t-(n.getTimezoneOffset()-t.getTimezoneOffset())*Qg)/ny))}const xy=my(0),wy=my(1),My=my(2),Ty=my(3),Ay=my(4),Sy=my(5),Ey=my(6),Ny=xy.range,ky=wy.range,Cy=My.range,Py=Ty.range,zy=Ay.range,$y=Sy.range,Dy=Ey.range;function Ry(t){return Vg((n=>{n.setUTCDate(n.getUTCDate()-(n.getUTCDay()+7-t)%7),n.setUTCHours(0,0,0,0)}),((t,n)=>{t.setUTCDate(t.getUTCDate()+7*n)}),((t,n)=>(n-t)/ny))}const Fy=Ry(0),qy=Ry(1),Uy=Ry(2),Iy=Ry(3),Oy=Ry(4),By=Ry(5),Yy=Ry(6),Ly=Fy.range,jy=qy.range,Hy=Uy.range,Xy=Iy.range,Gy=Oy.range,Vy=By.range,Wy=Yy.range,Zy=Vg((t=>{t.setDate(1),t.setHours(0,0,0,0)}),((t,n)=>{t.setMonth(t.getMonth()+n)}),((t,n)=>n.getMonth()-t.getMonth()+12*(n.getFullYear()-t.getFullYear())),(t=>t.getMonth())),Ky=Zy.range,Qy=Vg((t=>{t.setUTCDate(1),t.setUTCHours(0,0,0,0)}),((t,n)=>{t.setUTCMonth(t.getUTCMonth()+n)}),((t,n)=>n.getUTCMonth()-t.getUTCMonth()+12*(n.getUTCFullYear()-t.getUTCFullYear())),(t=>t.getUTCMonth())),Jy=Qy.range,tv=Vg((t=>{t.setMonth(0,1),t.setHours(0,0,0,0)}),((t,n)=>{t.setFullYear(t.getFullYear()+n)}),((t,n)=>n.getFullYear()-t.getFullYear()),(t=>t.getFullYear()));tv.every=t=>isFinite(t=Math.floor(t))&&t>0?Vg((n=>{n.setFullYear(Math.floor(n.getFullYear()/t)*t),n.setMonth(0,1),n.setHours(0,0,0,0)}),((n,e)=>{n.setFullYear(n.getFullYear()+e*t)})):null;const nv=tv.range,ev=Vg((t=>{t.setUTCMonth(0,1),t.setUTCHours(0,0,0,0)}),((t,n)=>{t.setUTCFullYear(t.getUTCFullYear()+n)}),((t,n)=>n.getUTCFullYear()-t.getUTCFullYear()),(t=>t.getUTCFullYear()));ev.every=t=>isFinite(t=Math.floor(t))&&t>0?Vg((n=>{n.setUTCFullYear(Math.floor(n.getUTCFullYear()/t)*t),n.setUTCMonth(0,1),n.setUTCHours(0,0,0,0)}),((n,e)=>{n.setUTCFullYear(n.getUTCFullYear()+e*t)})):null;const rv=ev.range;function iv(t,n,e,i,o,a){const u=[[iy,1,Kg],[iy,5,5e3],[iy,15,15e3],[iy,30,3e4],[a,1,Qg],[a,5,3e5],[a,15,9e5],[a,30,18e5],[o,1,Jg],[o,3,108e5],[o,6,216e5],[o,12,432e5],[i,1,ty],[i,2,1728e5],[e,1,ny],[n,1,ey],[n,3,7776e6],[t,1,ry]];function c(n,e,i){const o=Math.abs(e-n)/i,a=r((([,,t])=>t)).right(u,o);if(a===u.length)return t.every(W(n/ry,e/ry,i));if(0===a)return Wg.every(Math.max(W(n,e,i),1));const[c,f]=u[o/u[a-1][2]=12)]},q:function(t){return 1+~~(t.getMonth()/3)},Q:k_,s:C_,S:Zv,u:Kv,U:Qv,V:t_,w:n_,W:e_,x:null,X:null,y:r_,Y:o_,Z:u_,"%":N_},m={a:function(t){return a[t.getUTCDay()]},A:function(t){return o[t.getUTCDay()]},b:function(t){return c[t.getUTCMonth()]},B:function(t){return u[t.getUTCMonth()]},c:null,d:c_,e:c_,f:d_,g:T_,G:S_,H:f_,I:s_,j:l_,L:h_,m:p_,M:g_,p:function(t){return i[+(t.getUTCHours()>=12)]},q:function(t){return 1+~~(t.getUTCMonth()/3)},Q:k_,s:C_,S:y_,u:v_,U:__,V:m_,w:x_,W:w_,x:null,X:null,y:M_,Y:A_,Z:E_,"%":N_},x={a:function(t,n,e){var r=d.exec(n.slice(e));return r?(t.w=p.get(r[0].toLowerCase()),e+r[0].length):-1},A:function(t,n,e){var r=l.exec(n.slice(e));return r?(t.w=h.get(r[0].toLowerCase()),e+r[0].length):-1},b:function(t,n,e){var r=v.exec(n.slice(e));return r?(t.m=_.get(r[0].toLowerCase()),e+r[0].length):-1},B:function(t,n,e){var r=g.exec(n.slice(e));return r?(t.m=y.get(r[0].toLowerCase()),e+r[0].length):-1},c:function(t,e,r){return T(t,n,e,r)},d:zv,e:zv,f:Uv,g:Nv,G:Ev,H:Dv,I:Dv,j:$v,L:qv,m:Pv,M:Rv,p:function(t,n,e){var r=f.exec(n.slice(e));return r?(t.p=s.get(r[0].toLowerCase()),e+r[0].length):-1},q:Cv,Q:Ov,s:Bv,S:Fv,u:Mv,U:Tv,V:Av,w:wv,W:Sv,x:function(t,n,r){return T(t,e,n,r)},X:function(t,n,e){return T(t,r,n,e)},y:Nv,Y:Ev,Z:kv,"%":Iv};function w(t,n){return function(e){var r,i,o,a=[],u=-1,c=0,f=t.length;for(e instanceof Date||(e=new Date(+e));++u53)return null;"w"in o||(o.w=1),"Z"in o?(i=(r=sv(lv(o.y,0,1))).getUTCDay(),r=i>4||0===i?qy.ceil(r):qy(r),r=yy.offset(r,7*(o.V-1)),o.y=r.getUTCFullYear(),o.m=r.getUTCMonth(),o.d=r.getUTCDate()+(o.w+6)%7):(i=(r=fv(lv(o.y,0,1))).getDay(),r=i>4||0===i?wy.ceil(r):wy(r),r=py.offset(r,7*(o.V-1)),o.y=r.getFullYear(),o.m=r.getMonth(),o.d=r.getDate()+(o.w+6)%7)}else("W"in o||"U"in o)&&("w"in o||(o.w="u"in o?o.u%7:"W"in o?1:0),i="Z"in o?sv(lv(o.y,0,1)).getUTCDay():fv(lv(o.y,0,1)).getDay(),o.m=0,o.d="W"in o?(o.w+6)%7+7*o.W-(i+5)%7:o.w+7*o.U-(i+6)%7);return"Z"in o?(o.H+=o.Z/100|0,o.M+=o.Z%100,sv(o)):fv(o)}}function T(t,n,e,r){for(var i,o,a=0,u=n.length,c=e.length;a=c)return-1;if(37===(i=n.charCodeAt(a++))){if(i=n.charAt(a++),!(o=x[i in pv?n.charAt(a++):i])||(r=o(t,e,r))<0)return-1}else if(i!=e.charCodeAt(r++))return-1}return r}return b.x=w(e,b),b.X=w(r,b),b.c=w(n,b),m.x=w(e,m),m.X=w(r,m),m.c=w(n,m),{format:function(t){var n=w(t+="",b);return n.toString=function(){return t},n},parse:function(t){var n=M(t+="",!1);return n.toString=function(){return t},n},utcFormat:function(t){var n=w(t+="",m);return n.toString=function(){return t},n},utcParse:function(t){var n=M(t+="",!0);return n.toString=function(){return t},n}}}var dv,pv={"-":"",_:" ",0:"0"},gv=/^\s*\d+/,yv=/^%/,vv=/[\\^$*+?|[\]().{}]/g;function _v(t,n,e){var r=t<0?"-":"",i=(r?-t:t)+"",o=i.length;return r+(o[t.toLowerCase(),n])))}function wv(t,n,e){var r=gv.exec(n.slice(e,e+1));return r?(t.w=+r[0],e+r[0].length):-1}function Mv(t,n,e){var r=gv.exec(n.slice(e,e+1));return r?(t.u=+r[0],e+r[0].length):-1}function Tv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.U=+r[0],e+r[0].length):-1}function Av(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.V=+r[0],e+r[0].length):-1}function Sv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.W=+r[0],e+r[0].length):-1}function Ev(t,n,e){var r=gv.exec(n.slice(e,e+4));return r?(t.y=+r[0],e+r[0].length):-1}function Nv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.y=+r[0]+(+r[0]>68?1900:2e3),e+r[0].length):-1}function kv(t,n,e){var r=/^(Z)|([+-]\d\d)(?::?(\d\d))?/.exec(n.slice(e,e+6));return r?(t.Z=r[1]?0:-(r[2]+(r[3]||"00")),e+r[0].length):-1}function Cv(t,n,e){var r=gv.exec(n.slice(e,e+1));return r?(t.q=3*r[0]-3,e+r[0].length):-1}function Pv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.m=r[0]-1,e+r[0].length):-1}function zv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.d=+r[0],e+r[0].length):-1}function $v(t,n,e){var r=gv.exec(n.slice(e,e+3));return r?(t.m=0,t.d=+r[0],e+r[0].length):-1}function Dv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.H=+r[0],e+r[0].length):-1}function Rv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.M=+r[0],e+r[0].length):-1}function Fv(t,n,e){var r=gv.exec(n.slice(e,e+2));return r?(t.S=+r[0],e+r[0].length):-1}function qv(t,n,e){var r=gv.exec(n.slice(e,e+3));return r?(t.L=+r[0],e+r[0].length):-1}function Uv(t,n,e){var r=gv.exec(n.slice(e,e+6));return r?(t.L=Math.floor(r[0]/1e3),e+r[0].length):-1}function Iv(t,n,e){var r=yv.exec(n.slice(e,e+1));return r?e+r[0].length:-1}function Ov(t,n,e){var r=gv.exec(n.slice(e));return r?(t.Q=+r[0],e+r[0].length):-1}function Bv(t,n,e){var r=gv.exec(n.slice(e));return r?(t.s=+r[0],e+r[0].length):-1}function Yv(t,n){return _v(t.getDate(),n,2)}function Lv(t,n){return _v(t.getHours(),n,2)}function jv(t,n){return _v(t.getHours()%12||12,n,2)}function Hv(t,n){return _v(1+py.count(tv(t),t),n,3)}function Xv(t,n){return _v(t.getMilliseconds(),n,3)}function Gv(t,n){return Xv(t,n)+"000"}function Vv(t,n){return _v(t.getMonth()+1,n,2)}function Wv(t,n){return _v(t.getMinutes(),n,2)}function Zv(t,n){return _v(t.getSeconds(),n,2)}function Kv(t){var n=t.getDay();return 0===n?7:n}function Qv(t,n){return _v(xy.count(tv(t)-1,t),n,2)}function Jv(t){var n=t.getDay();return n>=4||0===n?Ay(t):Ay.ceil(t)}function t_(t,n){return t=Jv(t),_v(Ay.count(tv(t),t)+(4===tv(t).getDay()),n,2)}function n_(t){return t.getDay()}function e_(t,n){return _v(wy.count(tv(t)-1,t),n,2)}function r_(t,n){return _v(t.getFullYear()%100,n,2)}function i_(t,n){return _v((t=Jv(t)).getFullYear()%100,n,2)}function o_(t,n){return _v(t.getFullYear()%1e4,n,4)}function a_(t,n){var e=t.getDay();return _v((t=e>=4||0===e?Ay(t):Ay.ceil(t)).getFullYear()%1e4,n,4)}function u_(t){var n=t.getTimezoneOffset();return(n>0?"-":(n*=-1,"+"))+_v(n/60|0,"0",2)+_v(n%60,"0",2)}function c_(t,n){return _v(t.getUTCDate(),n,2)}function f_(t,n){return _v(t.getUTCHours(),n,2)}function s_(t,n){return _v(t.getUTCHours()%12||12,n,2)}function l_(t,n){return _v(1+yy.count(ev(t),t),n,3)}function h_(t,n){return _v(t.getUTCMilliseconds(),n,3)}function d_(t,n){return h_(t,n)+"000"}function p_(t,n){return _v(t.getUTCMonth()+1,n,2)}function g_(t,n){return _v(t.getUTCMinutes(),n,2)}function y_(t,n){return _v(t.getUTCSeconds(),n,2)}function v_(t){var n=t.getUTCDay();return 0===n?7:n}function __(t,n){return _v(Fy.count(ev(t)-1,t),n,2)}function b_(t){var n=t.getUTCDay();return n>=4||0===n?Oy(t):Oy.ceil(t)}function m_(t,n){return t=b_(t),_v(Oy.count(ev(t),t)+(4===ev(t).getUTCDay()),n,2)}function x_(t){return t.getUTCDay()}function w_(t,n){return _v(qy.count(ev(t)-1,t),n,2)}function M_(t,n){return _v(t.getUTCFullYear()%100,n,2)}function T_(t,n){return _v((t=b_(t)).getUTCFullYear()%100,n,2)}function A_(t,n){return _v(t.getUTCFullYear()%1e4,n,4)}function S_(t,n){var e=t.getUTCDay();return _v((t=e>=4||0===e?Oy(t):Oy.ceil(t)).getUTCFullYear()%1e4,n,4)}function E_(){return"+0000"}function N_(){return"%"}function k_(t){return+t}function C_(t){return Math.floor(+t/1e3)}function P_(n){return dv=hv(n),t.timeFormat=dv.format,t.timeParse=dv.parse,t.utcFormat=dv.utcFormat,t.utcParse=dv.utcParse,dv}t.timeFormat=void 0,t.timeParse=void 0,t.utcFormat=void 0,t.utcParse=void 0,P_({dateTime:"%x, %X",date:"%-m/%-d/%Y",time:"%-I:%M:%S %p",periods:["AM","PM"],days:["Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday"],shortDays:["Sun","Mon","Tue","Wed","Thu","Fri","Sat"],months:["January","February","March","April","May","June","July","August","September","October","November","December"],shortMonths:["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"]});var z_="%Y-%m-%dT%H:%M:%S.%LZ";var $_=Date.prototype.toISOString?function(t){return t.toISOString()}:t.utcFormat(z_),D_=$_;var R_=+new Date("2000-01-01T00:00:00.000Z")?function(t){var n=new Date(t);return isNaN(n)?null:n}:t.utcParse(z_),F_=R_;function q_(t){return new Date(t)}function U_(t){return t instanceof Date?+t:+new Date(+t)}function I_(t,n,e,r,i,o,a,u,c,f){var s=Sg(),l=s.invert,h=s.domain,d=f(".%L"),p=f(":%S"),g=f("%I:%M"),y=f("%I %p"),v=f("%a %d"),_=f("%b %d"),b=f("%B"),m=f("%Y");function x(t){return(c(t)Fr(t[t.length-1]),ib=new Array(3).concat("d8b365f5f5f55ab4ac","a6611adfc27d80cdc1018571","a6611adfc27df5f5f580cdc1018571","8c510ad8b365f6e8c3c7eae55ab4ac01665e","8c510ad8b365f6e8c3f5f5f5c7eae55ab4ac01665e","8c510abf812ddfc27df6e8c3c7eae580cdc135978f01665e","8c510abf812ddfc27df6e8c3f5f5f5c7eae580cdc135978f01665e","5430058c510abf812ddfc27df6e8c3c7eae580cdc135978f01665e003c30","5430058c510abf812ddfc27df6e8c3f5f5f5c7eae580cdc135978f01665e003c30").map(H_),ob=rb(ib),ab=new Array(3).concat("af8dc3f7f7f77fbf7b","7b3294c2a5cfa6dba0008837","7b3294c2a5cff7f7f7a6dba0008837","762a83af8dc3e7d4e8d9f0d37fbf7b1b7837","762a83af8dc3e7d4e8f7f7f7d9f0d37fbf7b1b7837","762a839970abc2a5cfe7d4e8d9f0d3a6dba05aae611b7837","762a839970abc2a5cfe7d4e8f7f7f7d9f0d3a6dba05aae611b7837","40004b762a839970abc2a5cfe7d4e8d9f0d3a6dba05aae611b783700441b","40004b762a839970abc2a5cfe7d4e8f7f7f7d9f0d3a6dba05aae611b783700441b").map(H_),ub=rb(ab),cb=new Array(3).concat("e9a3c9f7f7f7a1d76a","d01c8bf1b6dab8e1864dac26","d01c8bf1b6daf7f7f7b8e1864dac26","c51b7de9a3c9fde0efe6f5d0a1d76a4d9221","c51b7de9a3c9fde0eff7f7f7e6f5d0a1d76a4d9221","c51b7dde77aef1b6dafde0efe6f5d0b8e1867fbc414d9221","c51b7dde77aef1b6dafde0eff7f7f7e6f5d0b8e1867fbc414d9221","8e0152c51b7dde77aef1b6dafde0efe6f5d0b8e1867fbc414d9221276419","8e0152c51b7dde77aef1b6dafde0eff7f7f7e6f5d0b8e1867fbc414d9221276419").map(H_),fb=rb(cb),sb=new Array(3).concat("998ec3f7f7f7f1a340","5e3c99b2abd2fdb863e66101","5e3c99b2abd2f7f7f7fdb863e66101","542788998ec3d8daebfee0b6f1a340b35806","542788998ec3d8daebf7f7f7fee0b6f1a340b35806","5427888073acb2abd2d8daebfee0b6fdb863e08214b35806","5427888073acb2abd2d8daebf7f7f7fee0b6fdb863e08214b35806","2d004b5427888073acb2abd2d8daebfee0b6fdb863e08214b358067f3b08","2d004b5427888073acb2abd2d8daebf7f7f7fee0b6fdb863e08214b358067f3b08").map(H_),lb=rb(sb),hb=new Array(3).concat("ef8a62f7f7f767a9cf","ca0020f4a58292c5de0571b0","ca0020f4a582f7f7f792c5de0571b0","b2182bef8a62fddbc7d1e5f067a9cf2166ac","b2182bef8a62fddbc7f7f7f7d1e5f067a9cf2166ac","b2182bd6604df4a582fddbc7d1e5f092c5de4393c32166ac","b2182bd6604df4a582fddbc7f7f7f7d1e5f092c5de4393c32166ac","67001fb2182bd6604df4a582fddbc7d1e5f092c5de4393c32166ac053061","67001fb2182bd6604df4a582fddbc7f7f7f7d1e5f092c5de4393c32166ac053061").map(H_),db=rb(hb),pb=new Array(3).concat("ef8a62ffffff999999","ca0020f4a582bababa404040","ca0020f4a582ffffffbababa404040","b2182bef8a62fddbc7e0e0e09999994d4d4d","b2182bef8a62fddbc7ffffffe0e0e09999994d4d4d","b2182bd6604df4a582fddbc7e0e0e0bababa8787874d4d4d","b2182bd6604df4a582fddbc7ffffffe0e0e0bababa8787874d4d4d","67001fb2182bd6604df4a582fddbc7e0e0e0bababa8787874d4d4d1a1a1a","67001fb2182bd6604df4a582fddbc7ffffffe0e0e0bababa8787874d4d4d1a1a1a").map(H_),gb=rb(pb),yb=new Array(3).concat("fc8d59ffffbf91bfdb","d7191cfdae61abd9e92c7bb6","d7191cfdae61ffffbfabd9e92c7bb6","d73027fc8d59fee090e0f3f891bfdb4575b4","d73027fc8d59fee090ffffbfe0f3f891bfdb4575b4","d73027f46d43fdae61fee090e0f3f8abd9e974add14575b4","d73027f46d43fdae61fee090ffffbfe0f3f8abd9e974add14575b4","a50026d73027f46d43fdae61fee090e0f3f8abd9e974add14575b4313695","a50026d73027f46d43fdae61fee090ffffbfe0f3f8abd9e974add14575b4313695").map(H_),vb=rb(yb),_b=new Array(3).concat("fc8d59ffffbf91cf60","d7191cfdae61a6d96a1a9641","d7191cfdae61ffffbfa6d96a1a9641","d73027fc8d59fee08bd9ef8b91cf601a9850","d73027fc8d59fee08bffffbfd9ef8b91cf601a9850","d73027f46d43fdae61fee08bd9ef8ba6d96a66bd631a9850","d73027f46d43fdae61fee08bffffbfd9ef8ba6d96a66bd631a9850","a50026d73027f46d43fdae61fee08bd9ef8ba6d96a66bd631a9850006837","a50026d73027f46d43fdae61fee08bffffbfd9ef8ba6d96a66bd631a9850006837").map(H_),bb=rb(_b),mb=new Array(3).concat("fc8d59ffffbf99d594","d7191cfdae61abdda42b83ba","d7191cfdae61ffffbfabdda42b83ba","d53e4ffc8d59fee08be6f59899d5943288bd","d53e4ffc8d59fee08bffffbfe6f59899d5943288bd","d53e4ff46d43fdae61fee08be6f598abdda466c2a53288bd","d53e4ff46d43fdae61fee08bffffbfe6f598abdda466c2a53288bd","9e0142d53e4ff46d43fdae61fee08be6f598abdda466c2a53288bd5e4fa2","9e0142d53e4ff46d43fdae61fee08bffffbfe6f598abdda466c2a53288bd5e4fa2").map(H_),xb=rb(mb),wb=new Array(3).concat("e5f5f999d8c92ca25f","edf8fbb2e2e266c2a4238b45","edf8fbb2e2e266c2a42ca25f006d2c","edf8fbccece699d8c966c2a42ca25f006d2c","edf8fbccece699d8c966c2a441ae76238b45005824","f7fcfde5f5f9ccece699d8c966c2a441ae76238b45005824","f7fcfde5f5f9ccece699d8c966c2a441ae76238b45006d2c00441b").map(H_),Mb=rb(wb),Tb=new Array(3).concat("e0ecf49ebcda8856a7","edf8fbb3cde38c96c688419d","edf8fbb3cde38c96c68856a7810f7c","edf8fbbfd3e69ebcda8c96c68856a7810f7c","edf8fbbfd3e69ebcda8c96c68c6bb188419d6e016b","f7fcfde0ecf4bfd3e69ebcda8c96c68c6bb188419d6e016b","f7fcfde0ecf4bfd3e69ebcda8c96c68c6bb188419d810f7c4d004b").map(H_),Ab=rb(Tb),Sb=new Array(3).concat("e0f3dba8ddb543a2ca","f0f9e8bae4bc7bccc42b8cbe","f0f9e8bae4bc7bccc443a2ca0868ac","f0f9e8ccebc5a8ddb57bccc443a2ca0868ac","f0f9e8ccebc5a8ddb57bccc44eb3d32b8cbe08589e","f7fcf0e0f3dbccebc5a8ddb57bccc44eb3d32b8cbe08589e","f7fcf0e0f3dbccebc5a8ddb57bccc44eb3d32b8cbe0868ac084081").map(H_),Eb=rb(Sb),Nb=new Array(3).concat("fee8c8fdbb84e34a33","fef0d9fdcc8afc8d59d7301f","fef0d9fdcc8afc8d59e34a33b30000","fef0d9fdd49efdbb84fc8d59e34a33b30000","fef0d9fdd49efdbb84fc8d59ef6548d7301f990000","fff7ecfee8c8fdd49efdbb84fc8d59ef6548d7301f990000","fff7ecfee8c8fdd49efdbb84fc8d59ef6548d7301fb300007f0000").map(H_),kb=rb(Nb),Cb=new Array(3).concat("ece2f0a6bddb1c9099","f6eff7bdc9e167a9cf02818a","f6eff7bdc9e167a9cf1c9099016c59","f6eff7d0d1e6a6bddb67a9cf1c9099016c59","f6eff7d0d1e6a6bddb67a9cf3690c002818a016450","fff7fbece2f0d0d1e6a6bddb67a9cf3690c002818a016450","fff7fbece2f0d0d1e6a6bddb67a9cf3690c002818a016c59014636").map(H_),Pb=rb(Cb),zb=new Array(3).concat("ece7f2a6bddb2b8cbe","f1eef6bdc9e174a9cf0570b0","f1eef6bdc9e174a9cf2b8cbe045a8d","f1eef6d0d1e6a6bddb74a9cf2b8cbe045a8d","f1eef6d0d1e6a6bddb74a9cf3690c00570b0034e7b","fff7fbece7f2d0d1e6a6bddb74a9cf3690c00570b0034e7b","fff7fbece7f2d0d1e6a6bddb74a9cf3690c00570b0045a8d023858").map(H_),$b=rb(zb),Db=new Array(3).concat("e7e1efc994c7dd1c77","f1eef6d7b5d8df65b0ce1256","f1eef6d7b5d8df65b0dd1c77980043","f1eef6d4b9dac994c7df65b0dd1c77980043","f1eef6d4b9dac994c7df65b0e7298ace125691003f","f7f4f9e7e1efd4b9dac994c7df65b0e7298ace125691003f","f7f4f9e7e1efd4b9dac994c7df65b0e7298ace125698004367001f").map(H_),Rb=rb(Db),Fb=new Array(3).concat("fde0ddfa9fb5c51b8a","feebe2fbb4b9f768a1ae017e","feebe2fbb4b9f768a1c51b8a7a0177","feebe2fcc5c0fa9fb5f768a1c51b8a7a0177","feebe2fcc5c0fa9fb5f768a1dd3497ae017e7a0177","fff7f3fde0ddfcc5c0fa9fb5f768a1dd3497ae017e7a0177","fff7f3fde0ddfcc5c0fa9fb5f768a1dd3497ae017e7a017749006a").map(H_),qb=rb(Fb),Ub=new Array(3).concat("edf8b17fcdbb2c7fb8","ffffcca1dab441b6c4225ea8","ffffcca1dab441b6c42c7fb8253494","ffffccc7e9b47fcdbb41b6c42c7fb8253494","ffffccc7e9b47fcdbb41b6c41d91c0225ea80c2c84","ffffd9edf8b1c7e9b47fcdbb41b6c41d91c0225ea80c2c84","ffffd9edf8b1c7e9b47fcdbb41b6c41d91c0225ea8253494081d58").map(H_),Ib=rb(Ub),Ob=new Array(3).concat("f7fcb9addd8e31a354","ffffccc2e69978c679238443","ffffccc2e69978c67931a354006837","ffffccd9f0a3addd8e78c67931a354006837","ffffccd9f0a3addd8e78c67941ab5d238443005a32","ffffe5f7fcb9d9f0a3addd8e78c67941ab5d238443005a32","ffffe5f7fcb9d9f0a3addd8e78c67941ab5d238443006837004529").map(H_),Bb=rb(Ob),Yb=new Array(3).concat("fff7bcfec44fd95f0e","ffffd4fed98efe9929cc4c02","ffffd4fed98efe9929d95f0e993404","ffffd4fee391fec44ffe9929d95f0e993404","ffffd4fee391fec44ffe9929ec7014cc4c028c2d04","ffffe5fff7bcfee391fec44ffe9929ec7014cc4c028c2d04","ffffe5fff7bcfee391fec44ffe9929ec7014cc4c02993404662506").map(H_),Lb=rb(Yb),jb=new Array(3).concat("ffeda0feb24cf03b20","ffffb2fecc5cfd8d3ce31a1c","ffffb2fecc5cfd8d3cf03b20bd0026","ffffb2fed976feb24cfd8d3cf03b20bd0026","ffffb2fed976feb24cfd8d3cfc4e2ae31a1cb10026","ffffccffeda0fed976feb24cfd8d3cfc4e2ae31a1cb10026","ffffccffeda0fed976feb24cfd8d3cfc4e2ae31a1cbd0026800026").map(H_),Hb=rb(jb),Xb=new Array(3).concat("deebf79ecae13182bd","eff3ffbdd7e76baed62171b5","eff3ffbdd7e76baed63182bd08519c","eff3ffc6dbef9ecae16baed63182bd08519c","eff3ffc6dbef9ecae16baed64292c62171b5084594","f7fbffdeebf7c6dbef9ecae16baed64292c62171b5084594","f7fbffdeebf7c6dbef9ecae16baed64292c62171b508519c08306b").map(H_),Gb=rb(Xb),Vb=new Array(3).concat("e5f5e0a1d99b31a354","edf8e9bae4b374c476238b45","edf8e9bae4b374c47631a354006d2c","edf8e9c7e9c0a1d99b74c47631a354006d2c","edf8e9c7e9c0a1d99b74c47641ab5d238b45005a32","f7fcf5e5f5e0c7e9c0a1d99b74c47641ab5d238b45005a32","f7fcf5e5f5e0c7e9c0a1d99b74c47641ab5d238b45006d2c00441b").map(H_),Wb=rb(Vb),Zb=new Array(3).concat("f0f0f0bdbdbd636363","f7f7f7cccccc969696525252","f7f7f7cccccc969696636363252525","f7f7f7d9d9d9bdbdbd969696636363252525","f7f7f7d9d9d9bdbdbd969696737373525252252525","fffffff0f0f0d9d9d9bdbdbd969696737373525252252525","fffffff0f0f0d9d9d9bdbdbd969696737373525252252525000000").map(H_),Kb=rb(Zb),Qb=new Array(3).concat("efedf5bcbddc756bb1","f2f0f7cbc9e29e9ac86a51a3","f2f0f7cbc9e29e9ac8756bb154278f","f2f0f7dadaebbcbddc9e9ac8756bb154278f","f2f0f7dadaebbcbddc9e9ac8807dba6a51a34a1486","fcfbfdefedf5dadaebbcbddc9e9ac8807dba6a51a34a1486","fcfbfdefedf5dadaebbcbddc9e9ac8807dba6a51a354278f3f007d").map(H_),Jb=rb(Qb),tm=new Array(3).concat("fee0d2fc9272de2d26","fee5d9fcae91fb6a4acb181d","fee5d9fcae91fb6a4ade2d26a50f15","fee5d9fcbba1fc9272fb6a4ade2d26a50f15","fee5d9fcbba1fc9272fb6a4aef3b2ccb181d99000d","fff5f0fee0d2fcbba1fc9272fb6a4aef3b2ccb181d99000d","fff5f0fee0d2fcbba1fc9272fb6a4aef3b2ccb181da50f1567000d").map(H_),nm=rb(tm),em=new Array(3).concat("fee6cefdae6be6550d","feeddefdbe85fd8d3cd94701","feeddefdbe85fd8d3ce6550da63603","feeddefdd0a2fdae6bfd8d3ce6550da63603","feeddefdd0a2fdae6bfd8d3cf16913d948018c2d04","fff5ebfee6cefdd0a2fdae6bfd8d3cf16913d948018c2d04","fff5ebfee6cefdd0a2fdae6bfd8d3cf16913d94801a636037f2704").map(H_),rm=rb(em);var im=hi(Tr(300,.5,0),Tr(-240,.5,1)),om=hi(Tr(-100,.75,.35),Tr(80,1.5,.8)),am=hi(Tr(260,.75,.35),Tr(80,1.5,.8)),um=Tr();var cm=Fe(),fm=Math.PI/3,sm=2*Math.PI/3;function lm(t){var n=t.length;return function(e){return t[Math.max(0,Math.min(n-1,Math.floor(e*n)))]}}var hm=lm(H_("44015444025645045745055946075a46085c460a5d460b5e470d60470e6147106347116447136548146748166848176948186a481a6c481b6d481c6e481d6f481f70482071482173482374482475482576482677482878482979472a7a472c7a472d7b472e7c472f7d46307e46327e46337f463480453581453781453882443983443a83443b84433d84433e85423f854240864241864142874144874045884046883f47883f48893e49893e4a893e4c8a3d4d8a3d4e8a3c4f8a3c508b3b518b3b528b3a538b3a548c39558c39568c38588c38598c375a8c375b8d365c8d365d8d355e8d355f8d34608d34618d33628d33638d32648e32658e31668e31678e31688e30698e306a8e2f6b8e2f6c8e2e6d8e2e6e8e2e6f8e2d708e2d718e2c718e2c728e2c738e2b748e2b758e2a768e2a778e2a788e29798e297a8e297b8e287c8e287d8e277e8e277f8e27808e26818e26828e26828e25838e25848e25858e24868e24878e23888e23898e238a8d228b8d228c8d228d8d218e8d218f8d21908d21918c20928c20928c20938c1f948c1f958b1f968b1f978b1f988b1f998a1f9a8a1e9b8a1e9c891e9d891f9e891f9f881fa0881fa1881fa1871fa28720a38620a48621a58521a68522a78522a88423a98324aa8325ab8225ac8226ad8127ad8128ae8029af7f2ab07f2cb17e2db27d2eb37c2fb47c31b57b32b67a34b67935b77937b87838b9773aba763bbb753dbc743fbc7340bd7242be7144bf7046c06f48c16e4ac16d4cc26c4ec36b50c46a52c56954c56856c66758c7655ac8645cc8635ec96260ca6063cb5f65cb5e67cc5c69cd5b6ccd5a6ece5870cf5773d05675d05477d1537ad1517cd2507fd34e81d34d84d44b86d54989d5488bd6468ed64590d74393d74195d84098d83e9bd93c9dd93ba0da39a2da37a5db36a8db34aadc32addc30b0dd2fb2dd2db5de2bb8de29bade28bddf26c0df25c2df23c5e021c8e020cae11fcde11dd0e11cd2e21bd5e21ad8e219dae319dde318dfe318e2e418e5e419e7e419eae51aece51befe51cf1e51df4e61ef6e620f8e621fbe723fde725")),dm=lm(H_("00000401000501010601010802010902020b02020d03030f03031204041405041606051806051a07061c08071e0907200a08220b09240c09260d0a290e0b2b100b2d110c2f120d31130d34140e36150e38160f3b180f3d19103f1a10421c10441d11471e114920114b21114e22115024125325125527125829115a2a115c2c115f2d11612f116331116533106734106936106b38106c390f6e3b0f703d0f713f0f72400f74420f75440f764510774710784910784a10794c117a4e117b4f127b51127c52137c54137d56147d57157e59157e5a167e5c167f5d177f5f187f601880621980641a80651a80671b80681c816a1c816b1d816d1d816e1e81701f81721f817320817521817621817822817922827b23827c23827e24828025828125818326818426818627818827818928818b29818c29818e2a81902a81912b81932b80942c80962c80982d80992d809b2e7f9c2e7f9e2f7fa02f7fa1307ea3307ea5317ea6317da8327daa337dab337cad347cae347bb0357bb2357bb3367ab5367ab73779b83779ba3878bc3978bd3977bf3a77c03a76c23b75c43c75c53c74c73d73c83e73ca3e72cc3f71cd4071cf4070d0416fd2426fd3436ed5446dd6456cd8456cd9466bdb476adc4869de4968df4a68e04c67e24d66e34e65e44f64e55064e75263e85362e95462ea5661eb5760ec5860ed5a5fee5b5eef5d5ef05f5ef1605df2625df2645cf3655cf4675cf4695cf56b5cf66c5cf66e5cf7705cf7725cf8745cf8765cf9785df9795df97b5dfa7d5efa7f5efa815ffb835ffb8560fb8761fc8961fc8a62fc8c63fc8e64fc9065fd9266fd9467fd9668fd9869fd9a6afd9b6bfe9d6cfe9f6dfea16efea36ffea571fea772fea973feaa74feac76feae77feb078feb27afeb47bfeb67cfeb77efeb97ffebb81febd82febf84fec185fec287fec488fec68afec88cfeca8dfecc8ffecd90fecf92fed194fed395fed597fed799fed89afdda9cfddc9efddea0fde0a1fde2a3fde3a5fde5a7fde7a9fde9aafdebacfcecaefceeb0fcf0b2fcf2b4fcf4b6fcf6b8fcf7b9fcf9bbfcfbbdfcfdbf")),pm=lm(H_("00000401000501010601010802010a02020c02020e03021004031204031405041706041907051b08051d09061f0a07220b07240c08260d08290e092b10092d110a30120a32140b34150b37160b39180c3c190c3e1b0c411c0c431e0c451f0c48210c4a230c4c240c4f260c51280b53290b552b0b572d0b592f0a5b310a5c320a5e340a5f3609613809623909633b09643d09653e0966400a67420a68440a68450a69470b6a490b6a4a0c6b4c0c6b4d0d6c4f0d6c510e6c520e6d540f6d550f6d57106e59106e5a116e5c126e5d126e5f136e61136e62146e64156e65156e67166e69166e6a176e6c186e6d186e6f196e71196e721a6e741a6e751b6e771c6d781c6d7a1d6d7c1d6d7d1e6d7f1e6c801f6c82206c84206b85216b87216b88226a8a226a8c23698d23698f24699025689225689326679526679727669827669a28659b29649d29649f2a63a02a63a22b62a32c61a52c60a62d60a82e5fa92e5eab2f5ead305dae305cb0315bb1325ab3325ab43359b63458b73557b93556ba3655bc3754bd3853bf3952c03a51c13a50c33b4fc43c4ec63d4dc73e4cc83f4bca404acb4149cc4248ce4347cf4446d04545d24644d34743d44842d54a41d74b3fd84c3ed94d3dda4e3cdb503bdd513ade5238df5337e05536e15635e25734e35933e45a31e55c30e65d2fe75e2ee8602de9612bea632aeb6429eb6628ec6726ed6925ee6a24ef6c23ef6e21f06f20f1711ff1731df2741cf3761bf37819f47918f57b17f57d15f67e14f68013f78212f78410f8850ff8870ef8890cf98b0bf98c0af98e09fa9008fa9207fa9407fb9606fb9706fb9906fb9b06fb9d07fc9f07fca108fca309fca50afca60cfca80dfcaa0ffcac11fcae12fcb014fcb216fcb418fbb61afbb81dfbba1ffbbc21fbbe23fac026fac228fac42afac62df9c72ff9c932f9cb35f8cd37f8cf3af7d13df7d340f6d543f6d746f5d949f5db4cf4dd4ff4df53f4e156f3e35af3e55df2e661f2e865f2ea69f1ec6df1ed71f1ef75f1f179f2f27df2f482f3f586f3f68af4f88ef5f992f6fa96f8fb9af9fc9dfafda1fcffa4")),gm=lm(H_("0d088710078813078916078a19068c1b068d1d068e20068f2206902406912605912805922a05932c05942e05952f059631059733059735049837049938049a3a049a3c049b3e049c3f049c41049d43039e44039e46039f48039f4903a04b03a14c02a14e02a25002a25102a35302a35502a45601a45801a45901a55b01a55c01a65e01a66001a66100a76300a76400a76600a76700a86900a86a00a86c00a86e00a86f00a87100a87201a87401a87501a87701a87801a87a02a87b02a87d03a87e03a88004a88104a78305a78405a78606a68707a68808a68a09a58b0aa58d0ba58e0ca48f0da4910ea3920fa39410a29511a19613a19814a099159f9a169f9c179e9d189d9e199da01a9ca11b9ba21d9aa31e9aa51f99a62098a72197a82296aa2395ab2494ac2694ad2793ae2892b02991b12a90b22b8fb32c8eb42e8db52f8cb6308bb7318ab83289ba3388bb3488bc3587bd3786be3885bf3984c03a83c13b82c23c81c33d80c43e7fc5407ec6417dc7427cc8437bc9447aca457acb4679cc4778cc4977cd4a76ce4b75cf4c74d04d73d14e72d24f71d35171d45270d5536fd5546ed6556dd7566cd8576bd9586ada5a6ada5b69db5c68dc5d67dd5e66de5f65de6164df6263e06363e16462e26561e26660e3685fe4695ee56a5de56b5de66c5ce76e5be76f5ae87059e97158e97257ea7457eb7556eb7655ec7754ed7953ed7a52ee7b51ef7c51ef7e50f07f4ff0804ef1814df1834cf2844bf3854bf3874af48849f48948f58b47f58c46f68d45f68f44f79044f79143f79342f89441f89540f9973ff9983ef99a3efa9b3dfa9c3cfa9e3bfb9f3afba139fba238fca338fca537fca636fca835fca934fdab33fdac33fdae32fdaf31fdb130fdb22ffdb42ffdb52efeb72dfeb82cfeba2cfebb2bfebd2afebe2afec029fdc229fdc328fdc527fdc627fdc827fdca26fdcb26fccd25fcce25fcd025fcd225fbd324fbd524fbd724fad824fada24f9dc24f9dd25f8df25f8e125f7e225f7e425f6e626f6e826f5e926f5eb27f4ed27f3ee27f3f027f2f227f1f426f1f525f0f724f0f921"));function ym(t){return function(){return t}}const vm=Math.abs,_m=Math.atan2,bm=Math.cos,mm=Math.max,xm=Math.min,wm=Math.sin,Mm=Math.sqrt,Tm=1e-12,Am=Math.PI,Sm=Am/2,Em=2*Am;function Nm(t){return t>=1?Sm:t<=-1?-Sm:Math.asin(t)}function km(t){let n=3;return t.digits=function(e){if(!arguments.length)return n;if(null==e)n=null;else{const t=Math.floor(e);if(!(t>=0))throw new RangeError(`invalid digits: ${e}`);n=t}return t},()=>new Ua(n)}function Cm(t){return t.innerRadius}function Pm(t){return t.outerRadius}function zm(t){return t.startAngle}function $m(t){return t.endAngle}function Dm(t){return t&&t.padAngle}function Rm(t,n,e,r,i,o,a){var u=t-e,c=n-r,f=(a?o:-o)/Mm(u*u+c*c),s=f*c,l=-f*u,h=t+s,d=n+l,p=e+s,g=r+l,y=(h+p)/2,v=(d+g)/2,_=p-h,b=g-d,m=_*_+b*b,x=i-o,w=h*g-p*d,M=(b<0?-1:1)*Mm(mm(0,x*x*m-w*w)),T=(w*b-_*M)/m,A=(-w*_-b*M)/m,S=(w*b+_*M)/m,E=(-w*_+b*M)/m,N=T-y,k=A-v,C=S-y,P=E-v;return N*N+k*k>C*C+P*P&&(T=S,A=E),{cx:T,cy:A,x01:-s,y01:-l,x11:T*(i/x-1),y11:A*(i/x-1)}}var Fm=Array.prototype.slice;function qm(t){return"object"==typeof t&&"length"in t?t:Array.from(t)}function Um(t){this._context=t}function Im(t){return new Um(t)}function Om(t){return t[0]}function Bm(t){return t[1]}function Ym(t,n){var e=ym(!0),r=null,i=Im,o=null,a=km(u);function u(u){var c,f,s,l=(u=qm(u)).length,h=!1;for(null==r&&(o=i(s=a())),c=0;c<=l;++c)!(c=l;--h)u.point(v[h],_[h]);u.lineEnd(),u.areaEnd()}y&&(v[s]=+t(d,s,f),_[s]=+n(d,s,f),u.point(r?+r(d,s,f):v[s],e?+e(d,s,f):_[s]))}if(p)return u=null,p+""||null}function s(){return Ym().defined(i).curve(a).context(o)}return t="function"==typeof t?t:void 0===t?Om:ym(+t),n="function"==typeof n?n:ym(void 0===n?0:+n),e="function"==typeof e?e:void 0===e?Bm:ym(+e),f.x=function(n){return arguments.length?(t="function"==typeof n?n:ym(+n),r=null,f):t},f.x0=function(n){return arguments.length?(t="function"==typeof n?n:ym(+n),f):t},f.x1=function(t){return arguments.length?(r=null==t?null:"function"==typeof t?t:ym(+t),f):r},f.y=function(t){return arguments.length?(n="function"==typeof t?t:ym(+t),e=null,f):n},f.y0=function(t){return arguments.length?(n="function"==typeof t?t:ym(+t),f):n},f.y1=function(t){return arguments.length?(e=null==t?null:"function"==typeof t?t:ym(+t),f):e},f.lineX0=f.lineY0=function(){return s().x(t).y(n)},f.lineY1=function(){return s().x(t).y(e)},f.lineX1=function(){return s().x(r).y(n)},f.defined=function(t){return arguments.length?(i="function"==typeof t?t:ym(!!t),f):i},f.curve=function(t){return arguments.length?(a=t,null!=o&&(u=a(o)),f):a},f.context=function(t){return arguments.length?(null==t?o=u=null:u=a(o=t),f):o},f}function jm(t,n){return nt?1:n>=t?0:NaN}function Hm(t){return t}Um.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._point=0},lineEnd:function(){(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,n):this._context.moveTo(t,n);break;case 1:this._point=2;default:this._context.lineTo(t,n)}}};var Xm=Vm(Im);function Gm(t){this._curve=t}function Vm(t){function n(n){return new Gm(t(n))}return n._curve=t,n}function Wm(t){var n=t.curve;return t.angle=t.x,delete t.x,t.radius=t.y,delete t.y,t.curve=function(t){return arguments.length?n(Vm(t)):n()._curve},t}function Zm(){return Wm(Ym().curve(Xm))}function Km(){var t=Lm().curve(Xm),n=t.curve,e=t.lineX0,r=t.lineX1,i=t.lineY0,o=t.lineY1;return t.angle=t.x,delete t.x,t.startAngle=t.x0,delete t.x0,t.endAngle=t.x1,delete t.x1,t.radius=t.y,delete t.y,t.innerRadius=t.y0,delete t.y0,t.outerRadius=t.y1,delete t.y1,t.lineStartAngle=function(){return Wm(e())},delete t.lineX0,t.lineEndAngle=function(){return Wm(r())},delete t.lineX1,t.lineInnerRadius=function(){return Wm(i())},delete t.lineY0,t.lineOuterRadius=function(){return Wm(o())},delete t.lineY1,t.curve=function(t){return arguments.length?n(Vm(t)):n()._curve},t}function Qm(t,n){return[(n=+n)*Math.cos(t-=Math.PI/2),n*Math.sin(t)]}Gm.prototype={areaStart:function(){this._curve.areaStart()},areaEnd:function(){this._curve.areaEnd()},lineStart:function(){this._curve.lineStart()},lineEnd:function(){this._curve.lineEnd()},point:function(t,n){this._curve.point(n*Math.sin(t),n*-Math.cos(t))}};class Jm{constructor(t,n){this._context=t,this._x=n}areaStart(){this._line=0}areaEnd(){this._line=NaN}lineStart(){this._point=0}lineEnd(){(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line}point(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,n):this._context.moveTo(t,n);break;case 1:this._point=2;default:this._x?this._context.bezierCurveTo(this._x0=(this._x0+t)/2,this._y0,this._x0,n,t,n):this._context.bezierCurveTo(this._x0,this._y0=(this._y0+n)/2,t,this._y0,t,n)}this._x0=t,this._y0=n}}class tx{constructor(t){this._context=t}lineStart(){this._point=0}lineEnd(){}point(t,n){if(t=+t,n=+n,0===this._point)this._point=1;else{const e=Qm(this._x0,this._y0),r=Qm(this._x0,this._y0=(this._y0+n)/2),i=Qm(t,this._y0),o=Qm(t,n);this._context.moveTo(...e),this._context.bezierCurveTo(...r,...i,...o)}this._x0=t,this._y0=n}}function nx(t){return new Jm(t,!0)}function ex(t){return new Jm(t,!1)}function rx(t){return new tx(t)}function ix(t){return t.source}function ox(t){return t.target}function ax(t){let n=ix,e=ox,r=Om,i=Bm,o=null,a=null,u=km(c);function c(){let c;const f=Fm.call(arguments),s=n.apply(this,f),l=e.apply(this,f);if(null==o&&(a=t(c=u())),a.lineStart(),f[0]=s,a.point(+r.apply(this,f),+i.apply(this,f)),f[0]=l,a.point(+r.apply(this,f),+i.apply(this,f)),a.lineEnd(),c)return a=null,c+""||null}return c.source=function(t){return arguments.length?(n=t,c):n},c.target=function(t){return arguments.length?(e=t,c):e},c.x=function(t){return arguments.length?(r="function"==typeof t?t:ym(+t),c):r},c.y=function(t){return arguments.length?(i="function"==typeof t?t:ym(+t),c):i},c.context=function(n){return arguments.length?(null==n?o=a=null:a=t(o=n),c):o},c}const ux=Mm(3);var cx={draw(t,n){const e=.59436*Mm(n+xm(n/28,.75)),r=e/2,i=r*ux;t.moveTo(0,e),t.lineTo(0,-e),t.moveTo(-i,-r),t.lineTo(i,r),t.moveTo(-i,r),t.lineTo(i,-r)}},fx={draw(t,n){const e=Mm(n/Am);t.moveTo(e,0),t.arc(0,0,e,0,Em)}},sx={draw(t,n){const e=Mm(n/5)/2;t.moveTo(-3*e,-e),t.lineTo(-e,-e),t.lineTo(-e,-3*e),t.lineTo(e,-3*e),t.lineTo(e,-e),t.lineTo(3*e,-e),t.lineTo(3*e,e),t.lineTo(e,e),t.lineTo(e,3*e),t.lineTo(-e,3*e),t.lineTo(-e,e),t.lineTo(-3*e,e),t.closePath()}};const lx=Mm(1/3),hx=2*lx;var dx={draw(t,n){const e=Mm(n/hx),r=e*lx;t.moveTo(0,-e),t.lineTo(r,0),t.lineTo(0,e),t.lineTo(-r,0),t.closePath()}},px={draw(t,n){const e=.62625*Mm(n);t.moveTo(0,-e),t.lineTo(e,0),t.lineTo(0,e),t.lineTo(-e,0),t.closePath()}},gx={draw(t,n){const e=.87559*Mm(n-xm(n/7,2));t.moveTo(-e,0),t.lineTo(e,0),t.moveTo(0,e),t.lineTo(0,-e)}},yx={draw(t,n){const e=Mm(n),r=-e/2;t.rect(r,r,e,e)}},vx={draw(t,n){const e=.4431*Mm(n);t.moveTo(e,e),t.lineTo(e,-e),t.lineTo(-e,-e),t.lineTo(-e,e),t.closePath()}};const _x=wm(Am/10)/wm(7*Am/10),bx=wm(Em/10)*_x,mx=-bm(Em/10)*_x;var xx={draw(t,n){const e=Mm(.8908130915292852*n),r=bx*e,i=mx*e;t.moveTo(0,-e),t.lineTo(r,i);for(let n=1;n<5;++n){const o=Em*n/5,a=bm(o),u=wm(o);t.lineTo(u*e,-a*e),t.lineTo(a*r-u*i,u*r+a*i)}t.closePath()}};const wx=Mm(3);var Mx={draw(t,n){const e=-Mm(n/(3*wx));t.moveTo(0,2*e),t.lineTo(-wx*e,-e),t.lineTo(wx*e,-e),t.closePath()}};const Tx=Mm(3);var Ax={draw(t,n){const e=.6824*Mm(n),r=e/2,i=e*Tx/2;t.moveTo(0,-e),t.lineTo(i,r),t.lineTo(-i,r),t.closePath()}};const Sx=-.5,Ex=Mm(3)/2,Nx=1/Mm(12),kx=3*(Nx/2+1);var Cx={draw(t,n){const e=Mm(n/kx),r=e/2,i=e*Nx,o=r,a=e*Nx+e,u=-o,c=a;t.moveTo(r,i),t.lineTo(o,a),t.lineTo(u,c),t.lineTo(Sx*r-Ex*i,Ex*r+Sx*i),t.lineTo(Sx*o-Ex*a,Ex*o+Sx*a),t.lineTo(Sx*u-Ex*c,Ex*u+Sx*c),t.lineTo(Sx*r+Ex*i,Sx*i-Ex*r),t.lineTo(Sx*o+Ex*a,Sx*a-Ex*o),t.lineTo(Sx*u+Ex*c,Sx*c-Ex*u),t.closePath()}},Px={draw(t,n){const e=.6189*Mm(n-xm(n/6,1.7));t.moveTo(-e,-e),t.lineTo(e,e),t.moveTo(-e,e),t.lineTo(e,-e)}};const zx=[fx,sx,dx,yx,xx,Mx,Cx],$x=[fx,gx,Px,Ax,cx,vx,px];function Dx(){}function Rx(t,n,e){t._context.bezierCurveTo((2*t._x0+t._x1)/3,(2*t._y0+t._y1)/3,(t._x0+2*t._x1)/3,(t._y0+2*t._y1)/3,(t._x0+4*t._x1+n)/6,(t._y0+4*t._y1+e)/6)}function Fx(t){this._context=t}function qx(t){this._context=t}function Ux(t){this._context=t}function Ix(t,n){this._basis=new Fx(t),this._beta=n}Fx.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._y0=this._y1=NaN,this._point=0},lineEnd:function(){switch(this._point){case 3:Rx(this,this._x1,this._y1);case 2:this._context.lineTo(this._x1,this._y1)}(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,n):this._context.moveTo(t,n);break;case 1:this._point=2;break;case 2:this._point=3,this._context.lineTo((5*this._x0+this._x1)/6,(5*this._y0+this._y1)/6);default:Rx(this,t,n)}this._x0=this._x1,this._x1=t,this._y0=this._y1,this._y1=n}},qx.prototype={areaStart:Dx,areaEnd:Dx,lineStart:function(){this._x0=this._x1=this._x2=this._x3=this._x4=this._y0=this._y1=this._y2=this._y3=this._y4=NaN,this._point=0},lineEnd:function(){switch(this._point){case 1:this._context.moveTo(this._x2,this._y2),this._context.closePath();break;case 2:this._context.moveTo((this._x2+2*this._x3)/3,(this._y2+2*this._y3)/3),this._context.lineTo((this._x3+2*this._x2)/3,(this._y3+2*this._y2)/3),this._context.closePath();break;case 3:this.point(this._x2,this._y2),this.point(this._x3,this._y3),this.point(this._x4,this._y4)}},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1,this._x2=t,this._y2=n;break;case 1:this._point=2,this._x3=t,this._y3=n;break;case 2:this._point=3,this._x4=t,this._y4=n,this._context.moveTo((this._x0+4*this._x1+t)/6,(this._y0+4*this._y1+n)/6);break;default:Rx(this,t,n)}this._x0=this._x1,this._x1=t,this._y0=this._y1,this._y1=n}},Ux.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._y0=this._y1=NaN,this._point=0},lineEnd:function(){(this._line||0!==this._line&&3===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1;break;case 1:this._point=2;break;case 2:this._point=3;var e=(this._x0+4*this._x1+t)/6,r=(this._y0+4*this._y1+n)/6;this._line?this._context.lineTo(e,r):this._context.moveTo(e,r);break;case 3:this._point=4;default:Rx(this,t,n)}this._x0=this._x1,this._x1=t,this._y0=this._y1,this._y1=n}},Ix.prototype={lineStart:function(){this._x=[],this._y=[],this._basis.lineStart()},lineEnd:function(){var t=this._x,n=this._y,e=t.length-1;if(e>0)for(var r,i=t[0],o=n[0],a=t[e]-i,u=n[e]-o,c=-1;++c<=e;)r=c/e,this._basis.point(this._beta*t[c]+(1-this._beta)*(i+r*a),this._beta*n[c]+(1-this._beta)*(o+r*u));this._x=this._y=null,this._basis.lineEnd()},point:function(t,n){this._x.push(+t),this._y.push(+n)}};var Ox=function t(n){function e(t){return 1===n?new Fx(t):new Ix(t,n)}return e.beta=function(n){return t(+n)},e}(.85);function Bx(t,n,e){t._context.bezierCurveTo(t._x1+t._k*(t._x2-t._x0),t._y1+t._k*(t._y2-t._y0),t._x2+t._k*(t._x1-n),t._y2+t._k*(t._y1-e),t._x2,t._y2)}function Yx(t,n){this._context=t,this._k=(1-n)/6}Yx.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._point=0},lineEnd:function(){switch(this._point){case 2:this._context.lineTo(this._x2,this._y2);break;case 3:Bx(this,this._x1,this._y1)}(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,n):this._context.moveTo(t,n);break;case 1:this._point=2,this._x1=t,this._y1=n;break;case 2:this._point=3;default:Bx(this,t,n)}this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=n}};var Lx=function t(n){function e(t){return new Yx(t,n)}return e.tension=function(n){return t(+n)},e}(0);function jx(t,n){this._context=t,this._k=(1-n)/6}jx.prototype={areaStart:Dx,areaEnd:Dx,lineStart:function(){this._x0=this._x1=this._x2=this._x3=this._x4=this._x5=this._y0=this._y1=this._y2=this._y3=this._y4=this._y5=NaN,this._point=0},lineEnd:function(){switch(this._point){case 1:this._context.moveTo(this._x3,this._y3),this._context.closePath();break;case 2:this._context.lineTo(this._x3,this._y3),this._context.closePath();break;case 3:this.point(this._x3,this._y3),this.point(this._x4,this._y4),this.point(this._x5,this._y5)}},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1,this._x3=t,this._y3=n;break;case 1:this._point=2,this._context.moveTo(this._x4=t,this._y4=n);break;case 2:this._point=3,this._x5=t,this._y5=n;break;default:Bx(this,t,n)}this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=n}};var Hx=function t(n){function e(t){return new jx(t,n)}return e.tension=function(n){return t(+n)},e}(0);function Xx(t,n){this._context=t,this._k=(1-n)/6}Xx.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._point=0},lineEnd:function(){(this._line||0!==this._line&&3===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1;break;case 1:this._point=2;break;case 2:this._point=3,this._line?this._context.lineTo(this._x2,this._y2):this._context.moveTo(this._x2,this._y2);break;case 3:this._point=4;default:Bx(this,t,n)}this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=n}};var Gx=function t(n){function e(t){return new Xx(t,n)}return e.tension=function(n){return t(+n)},e}(0);function Vx(t,n,e){var r=t._x1,i=t._y1,o=t._x2,a=t._y2;if(t._l01_a>Tm){var u=2*t._l01_2a+3*t._l01_a*t._l12_a+t._l12_2a,c=3*t._l01_a*(t._l01_a+t._l12_a);r=(r*u-t._x0*t._l12_2a+t._x2*t._l01_2a)/c,i=(i*u-t._y0*t._l12_2a+t._y2*t._l01_2a)/c}if(t._l23_a>Tm){var f=2*t._l23_2a+3*t._l23_a*t._l12_a+t._l12_2a,s=3*t._l23_a*(t._l23_a+t._l12_a);o=(o*f+t._x1*t._l23_2a-n*t._l12_2a)/s,a=(a*f+t._y1*t._l23_2a-e*t._l12_2a)/s}t._context.bezierCurveTo(r,i,o,a,t._x2,t._y2)}function Wx(t,n){this._context=t,this._alpha=n}Wx.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._l01_a=this._l12_a=this._l23_a=this._l01_2a=this._l12_2a=this._l23_2a=this._point=0},lineEnd:function(){switch(this._point){case 2:this._context.lineTo(this._x2,this._y2);break;case 3:this.point(this._x2,this._y2)}(this._line||0!==this._line&&1===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,n){if(t=+t,n=+n,this._point){var e=this._x2-t,r=this._y2-n;this._l23_a=Math.sqrt(this._l23_2a=Math.pow(e*e+r*r,this._alpha))}switch(this._point){case 0:this._point=1,this._line?this._context.lineTo(t,n):this._context.moveTo(t,n);break;case 1:this._point=2;break;case 2:this._point=3;default:Vx(this,t,n)}this._l01_a=this._l12_a,this._l12_a=this._l23_a,this._l01_2a=this._l12_2a,this._l12_2a=this._l23_2a,this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=n}};var Zx=function t(n){function e(t){return n?new Wx(t,n):new Yx(t,0)}return e.alpha=function(n){return t(+n)},e}(.5);function Kx(t,n){this._context=t,this._alpha=n}Kx.prototype={areaStart:Dx,areaEnd:Dx,lineStart:function(){this._x0=this._x1=this._x2=this._x3=this._x4=this._x5=this._y0=this._y1=this._y2=this._y3=this._y4=this._y5=NaN,this._l01_a=this._l12_a=this._l23_a=this._l01_2a=this._l12_2a=this._l23_2a=this._point=0},lineEnd:function(){switch(this._point){case 1:this._context.moveTo(this._x3,this._y3),this._context.closePath();break;case 2:this._context.lineTo(this._x3,this._y3),this._context.closePath();break;case 3:this.point(this._x3,this._y3),this.point(this._x4,this._y4),this.point(this._x5,this._y5)}},point:function(t,n){if(t=+t,n=+n,this._point){var e=this._x2-t,r=this._y2-n;this._l23_a=Math.sqrt(this._l23_2a=Math.pow(e*e+r*r,this._alpha))}switch(this._point){case 0:this._point=1,this._x3=t,this._y3=n;break;case 1:this._point=2,this._context.moveTo(this._x4=t,this._y4=n);break;case 2:this._point=3,this._x5=t,this._y5=n;break;default:Vx(this,t,n)}this._l01_a=this._l12_a,this._l12_a=this._l23_a,this._l01_2a=this._l12_2a,this._l12_2a=this._l23_2a,this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=n}};var Qx=function t(n){function e(t){return n?new Kx(t,n):new jx(t,0)}return e.alpha=function(n){return t(+n)},e}(.5);function Jx(t,n){this._context=t,this._alpha=n}Jx.prototype={areaStart:function(){this._line=0},areaEnd:function(){this._line=NaN},lineStart:function(){this._x0=this._x1=this._x2=this._y0=this._y1=this._y2=NaN,this._l01_a=this._l12_a=this._l23_a=this._l01_2a=this._l12_2a=this._l23_2a=this._point=0},lineEnd:function(){(this._line||0!==this._line&&3===this._point)&&this._context.closePath(),this._line=1-this._line},point:function(t,n){if(t=+t,n=+n,this._point){var e=this._x2-t,r=this._y2-n;this._l23_a=Math.sqrt(this._l23_2a=Math.pow(e*e+r*r,this._alpha))}switch(this._point){case 0:this._point=1;break;case 1:this._point=2;break;case 2:this._point=3,this._line?this._context.lineTo(this._x2,this._y2):this._context.moveTo(this._x2,this._y2);break;case 3:this._point=4;default:Vx(this,t,n)}this._l01_a=this._l12_a,this._l12_a=this._l23_a,this._l01_2a=this._l12_2a,this._l12_2a=this._l23_2a,this._x0=this._x1,this._x1=this._x2,this._x2=t,this._y0=this._y1,this._y1=this._y2,this._y2=n}};var tw=function t(n){function e(t){return n?new Jx(t,n):new Xx(t,0)}return e.alpha=function(n){return t(+n)},e}(.5);function nw(t){this._context=t}function ew(t){return t<0?-1:1}function rw(t,n,e){var r=t._x1-t._x0,i=n-t._x1,o=(t._y1-t._y0)/(r||i<0&&-0),a=(e-t._y1)/(i||r<0&&-0),u=(o*i+a*r)/(r+i);return(ew(o)+ew(a))*Math.min(Math.abs(o),Math.abs(a),.5*Math.abs(u))||0}function iw(t,n){var e=t._x1-t._x0;return e?(3*(t._y1-t._y0)/e-n)/2:n}function ow(t,n,e){var r=t._x0,i=t._y0,o=t._x1,a=t._y1,u=(o-r)/3;t._context.bezierCurveTo(r+u,i+u*n,o-u,a-u*e,o,a)}function aw(t){this._context=t}function uw(t){this._context=new cw(t)}function cw(t){this._context=t}function fw(t){this._context=t}function sw(t){var n,e,r=t.length-1,i=new Array(r),o=new Array(r),a=new Array(r);for(i[0]=0,o[0]=2,a[0]=t[0]+2*t[1],n=1;n=0;--n)i[n]=(a[n]-i[n+1])/o[n];for(o[r-1]=(t[r]+i[r-1])/2,n=0;n1)for(var e,r,i,o=1,a=t[n[0]],u=a.length;o=0;)e[n]=n;return e}function pw(t,n){return t[n]}function gw(t){const n=[];return n.key=t,n}function yw(t){var n=t.map(vw);return dw(t).sort((function(t,e){return n[t]-n[e]}))}function vw(t){for(var n,e=-1,r=0,i=t.length,o=-1/0;++eo&&(o=n,r=e);return r}function _w(t){var n=t.map(bw);return dw(t).sort((function(t,e){return n[t]-n[e]}))}function bw(t){for(var n,e=0,r=-1,i=t.length;++r=0&&(this._t=1-this._t,this._line=1-this._line)},point:function(t,n){switch(t=+t,n=+n,this._point){case 0:this._point=1,this._line?this._context.lineTo(t,n):this._context.moveTo(t,n);break;case 1:this._point=2;default:if(this._t<=0)this._context.lineTo(this._x,n),this._context.lineTo(t,n);else{var e=this._x*(1-this._t)+t*this._t;this._context.lineTo(e,this._y),this._context.lineTo(e,n)}}this._x=t,this._y=n}};var mw=t=>()=>t;function xw(t,{sourceEvent:n,target:e,transform:r,dispatch:i}){Object.defineProperties(this,{type:{value:t,enumerable:!0,configurable:!0},sourceEvent:{value:n,enumerable:!0,configurable:!0},target:{value:e,enumerable:!0,configurable:!0},transform:{value:r,enumerable:!0,configurable:!0},_:{value:i}})}function ww(t,n,e){this.k=t,this.x=n,this.y=e}ww.prototype={constructor:ww,scale:function(t){return 1===t?this:new ww(this.k*t,this.x,this.y)},translate:function(t,n){return 0===t&0===n?this:new ww(this.k,this.x+this.k*t,this.y+this.k*n)},apply:function(t){return[t[0]*this.k+this.x,t[1]*this.k+this.y]},applyX:function(t){return t*this.k+this.x},applyY:function(t){return t*this.k+this.y},invert:function(t){return[(t[0]-this.x)/this.k,(t[1]-this.y)/this.k]},invertX:function(t){return(t-this.x)/this.k},invertY:function(t){return(t-this.y)/this.k},rescaleX:function(t){return t.copy().domain(t.range().map(this.invertX,this).map(t.invert,t))},rescaleY:function(t){return t.copy().domain(t.range().map(this.invertY,this).map(t.invert,t))},toString:function(){return"translate("+this.x+","+this.y+") scale("+this.k+")"}};var Mw=new ww(1,0,0);function Tw(t){for(;!t.__zoom;)if(!(t=t.parentNode))return Mw;return t.__zoom}function Aw(t){t.stopImmediatePropagation()}function Sw(t){t.preventDefault(),t.stopImmediatePropagation()}function Ew(t){return!(t.ctrlKey&&"wheel"!==t.type||t.button)}function Nw(){var t=this;return t instanceof SVGElement?(t=t.ownerSVGElement||t).hasAttribute("viewBox")?[[(t=t.viewBox.baseVal).x,t.y],[t.x+t.width,t.y+t.height]]:[[0,0],[t.width.baseVal.value,t.height.baseVal.value]]:[[0,0],[t.clientWidth,t.clientHeight]]}function kw(){return this.__zoom||Mw}function Cw(t){return-t.deltaY*(1===t.deltaMode?.05:t.deltaMode?1:.002)*(t.ctrlKey?10:1)}function Pw(){return navigator.maxTouchPoints||"ontouchstart"in this}function zw(t,n,e){var r=t.invertX(n[0][0])-e[0][0],i=t.invertX(n[1][0])-e[1][0],o=t.invertY(n[0][1])-e[0][1],a=t.invertY(n[1][1])-e[1][1];return t.translate(i>r?(r+i)/2:Math.min(0,r)||Math.max(0,i),a>o?(o+a)/2:Math.min(0,o)||Math.max(0,a))}Tw.prototype=ww.prototype,t.Adder=T,t.Delaunay=Lu,t.FormatSpecifier=tf,t.InternMap=InternMap,t.InternSet=InternSet,t.Node=Qd,t.Path=Ua,t.Voronoi=qu,t.ZoomTransform=ww,t.active=function(t,n){var e,r,i=t.__transition;if(i)for(r in n=null==n?null:n+"",i)if((e=i[r]).state>qi&&e.name===n)return new po([[t]],Zo,n,+r);return null},t.arc=function(){var t=Cm,n=Pm,e=ym(0),r=null,i=zm,o=$m,a=Dm,u=null,c=km(f);function f(){var f,s,l=+t.apply(this,arguments),h=+n.apply(this,arguments),d=i.apply(this,arguments)-Sm,p=o.apply(this,arguments)-Sm,g=vm(p-d),y=p>d;if(u||(u=f=c()),hTm)if(g>Em-Tm)u.moveTo(h*bm(d),h*wm(d)),u.arc(0,0,h,d,p,!y),l>Tm&&(u.moveTo(l*bm(p),l*wm(p)),u.arc(0,0,l,p,d,y));else{var v,_,b=d,m=p,x=d,w=p,M=g,T=g,A=a.apply(this,arguments)/2,S=A>Tm&&(r?+r.apply(this,arguments):Mm(l*l+h*h)),E=xm(vm(h-l)/2,+e.apply(this,arguments)),N=E,k=E;if(S>Tm){var C=Nm(S/l*wm(A)),P=Nm(S/h*wm(A));(M-=2*C)>Tm?(x+=C*=y?1:-1,w-=C):(M=0,x=w=(d+p)/2),(T-=2*P)>Tm?(b+=P*=y?1:-1,m-=P):(T=0,b=m=(d+p)/2)}var z=h*bm(b),$=h*wm(b),D=l*bm(w),R=l*wm(w);if(E>Tm){var F,q=h*bm(m),U=h*wm(m),I=l*bm(x),O=l*wm(x);if(g1?0:t<-1?Am:Math.acos(t)}((B*L+Y*j)/(Mm(B*B+Y*Y)*Mm(L*L+j*j)))/2),X=Mm(F[0]*F[0]+F[1]*F[1]);N=xm(E,(l-X)/(H-1)),k=xm(E,(h-X)/(H+1))}else N=k=0}T>Tm?k>Tm?(v=Rm(I,O,z,$,h,k,y),_=Rm(q,U,D,R,h,k,y),u.moveTo(v.cx+v.x01,v.cy+v.y01),kTm&&M>Tm?N>Tm?(v=Rm(D,R,q,U,l,-N,y),_=Rm(z,$,I,O,l,-N,y),u.lineTo(v.cx+v.x01,v.cy+v.y01),N=0))throw new RangeError("invalid r");let e=t.length;if(!((e=Math.floor(e))>=0))throw new RangeError("invalid length");if(!e||!n)return t;const r=y(n),i=t.slice();return r(t,i,0,e,1),r(i,t,0,e,1),r(t,i,0,e,1),t},t.blur2=l,t.blurImage=h,t.brush=function(){return wa(la)},t.brushSelection=function(t){var n=t.__brush;return n?n.dim.output(n.selection):null},t.brushX=function(){return wa(fa)},t.brushY=function(){return wa(sa)},t.buffer=function(t,n){return fetch(t,n).then(_c)},t.chord=function(){return za(!1,!1)},t.chordDirected=function(){return za(!0,!1)},t.chordTranspose=function(){return za(!1,!0)},t.cluster=function(){var t=Ld,n=1,e=1,r=!1;function i(i){var o,a=0;i.eachAfter((function(n){var e=n.children;e?(n.x=function(t){return t.reduce(jd,0)/t.length}(e),n.y=function(t){return 1+t.reduce(Hd,0)}(e)):(n.x=o?a+=t(n,o):0,n.y=0,o=n)}));var u=function(t){for(var n;n=t.children;)t=n[0];return t}(i),c=function(t){for(var n;n=t.children;)t=n[n.length-1];return t}(i),f=u.x-t(u,c)/2,s=c.x+t(c,u)/2;return i.eachAfter(r?function(t){t.x=(t.x-i.x)*n,t.y=(i.y-t.y)*e}:function(t){t.x=(t.x-f)/(s-f)*n,t.y=(1-(i.y?t.y/i.y:1))*e})}return i.separation=function(n){return arguments.length?(t=n,i):t},i.size=function(t){return arguments.length?(r=!1,n=+t[0],e=+t[1],i):r?null:[n,e]},i.nodeSize=function(t){return arguments.length?(r=!0,n=+t[0],e=+t[1],i):r?[n,e]:null},i},t.color=ze,t.contourDensity=function(){var t=fu,n=su,e=lu,r=960,i=500,o=20,a=2,u=3*o,c=r+2*u>>a,f=i+2*u>>a,s=Qa(20);function h(r){var i=new Float32Array(c*f),s=Math.pow(2,-a),h=-1;for(const o of r){var d=(t(o,++h,r)+u)*s,p=(n(o,h,r)+u)*s,g=+e(o,h,r);if(g&&d>=0&&d=0&&pt*r)))(n).map(((t,n)=>(t.value=+e[n],p(t))))}function p(t){return t.coordinates.forEach(g),t}function g(t){t.forEach(y)}function y(t){t.forEach(v)}function v(t){t[0]=t[0]*Math.pow(2,a)-u,t[1]=t[1]*Math.pow(2,a)-u}function _(){return c=r+2*(u=3*o)>>a,f=i+2*u>>a,d}return d.contours=function(t){var n=h(t),e=iu().size([c,f]),r=Math.pow(2,2*a),i=t=>{t=+t;var i=p(e.contour(n,t*r));return i.value=t,i};return Object.defineProperty(i,"max",{get:()=>J(n)/r}),i},d.x=function(n){return arguments.length?(t="function"==typeof n?n:Qa(+n),d):t},d.y=function(t){return arguments.length?(n="function"==typeof t?t:Qa(+t),d):n},d.weight=function(t){return arguments.length?(e="function"==typeof t?t:Qa(+t),d):e},d.size=function(t){if(!arguments.length)return[r,i];var n=+t[0],e=+t[1];if(!(n>=0&&e>=0))throw new Error("invalid size");return r=n,i=e,_()},d.cellSize=function(t){if(!arguments.length)return 1<=1))throw new Error("invalid cell size");return a=Math.floor(Math.log(t)/Math.LN2),_()},d.thresholds=function(t){return arguments.length?(s="function"==typeof t?t:Array.isArray(t)?Qa(Za.call(t)):Qa(t),d):s},d.bandwidth=function(t){if(!arguments.length)return Math.sqrt(o*(o+1));if(!((t=+t)>=0))throw new Error("invalid bandwidth");return o=(Math.sqrt(4*t*t+1)-1)/2,_()},d},t.contours=iu,t.count=v,t.create=function(t){return Zn(Yt(t).call(document.documentElement))},t.creator=Yt,t.cross=function(...t){const n="function"==typeof t[t.length-1]&&function(t){return n=>t(...n)}(t.pop()),e=(t=t.map(m)).map(_),r=t.length-1,i=new Array(r+1).fill(0),o=[];if(r<0||e.some(b))return o;for(;;){o.push(i.map(((n,e)=>t[e][n])));let a=r;for(;++i[a]===e[a];){if(0===a)return n?o.map(n):o;i[a--]=0}}},t.csv=wc,t.csvFormat=rc,t.csvFormatBody=ic,t.csvFormatRow=ac,t.csvFormatRows=oc,t.csvFormatValue=uc,t.csvParse=nc,t.csvParseRows=ec,t.cubehelix=Tr,t.cumsum=function(t,n){var e=0,r=0;return Float64Array.from(t,void 0===n?t=>e+=+t||0:i=>e+=+n(i,r++,t)||0)},t.curveBasis=function(t){return new Fx(t)},t.curveBasisClosed=function(t){return new qx(t)},t.curveBasisOpen=function(t){return new Ux(t)},t.curveBumpX=nx,t.curveBumpY=ex,t.curveBundle=Ox,t.curveCardinal=Lx,t.curveCardinalClosed=Hx,t.curveCardinalOpen=Gx,t.curveCatmullRom=Zx,t.curveCatmullRomClosed=Qx,t.curveCatmullRomOpen=tw,t.curveLinear=Im,t.curveLinearClosed=function(t){return new nw(t)},t.curveMonotoneX=function(t){return new aw(t)},t.curveMonotoneY=function(t){return new uw(t)},t.curveNatural=function(t){return new fw(t)},t.curveStep=function(t){return new lw(t,.5)},t.curveStepAfter=function(t){return new lw(t,1)},t.curveStepBefore=function(t){return new lw(t,0)},t.descending=e,t.deviation=w,t.difference=function(t,...n){t=new InternSet(t);for(const e of n)for(const n of e)t.delete(n);return t},t.disjoint=function(t,n){const e=n[Symbol.iterator](),r=new InternSet;for(const n of t){if(r.has(n))return!1;let t,i;for(;({value:t,done:i}=e.next())&&!i;){if(Object.is(n,t))return!1;r.add(t)}}return!0},t.dispatch=$t,t.drag=function(){var t,n,e,r,i=se,o=le,a=he,u=de,c={},f=$t("start","drag","end"),s=0,l=0;function h(t){t.on("mousedown.drag",d).filter(u).on("touchstart.drag",y).on("touchmove.drag",v,ee).on("touchend.drag touchcancel.drag",_).style("touch-action","none").style("-webkit-tap-highlight-color","rgba(0,0,0,0)")}function d(a,u){if(!r&&i.call(this,a,u)){var c=b(this,o.call(this,a,u),a,u,"mouse");c&&(Zn(a.view).on("mousemove.drag",p,re).on("mouseup.drag",g,re),ae(a.view),ie(a),e=!1,t=a.clientX,n=a.clientY,c("start",a))}}function p(r){if(oe(r),!e){var i=r.clientX-t,o=r.clientY-n;e=i*i+o*o>l}c.mouse("drag",r)}function g(t){Zn(t.view).on("mousemove.drag mouseup.drag",null),ue(t.view,e),oe(t),c.mouse("end",t)}function y(t,n){if(i.call(this,t,n)){var e,r,a=t.changedTouches,u=o.call(this,t,n),c=a.length;for(e=0;e+t,t.easePoly=wo,t.easePolyIn=mo,t.easePolyInOut=wo,t.easePolyOut=xo,t.easeQuad=_o,t.easeQuadIn=function(t){return t*t},t.easeQuadInOut=_o,t.easeQuadOut=function(t){return t*(2-t)},t.easeSin=Ao,t.easeSinIn=function(t){return 1==+t?1:1-Math.cos(t*To)},t.easeSinInOut=Ao,t.easeSinOut=function(t){return Math.sin(t*To)},t.every=function(t,n){if("function"!=typeof n)throw new TypeError("test is not a function");let e=-1;for(const r of t)if(!n(r,++e,t))return!1;return!0},t.extent=M,t.fcumsum=function(t,n){const e=new T;let r=-1;return Float64Array.from(t,void 0===n?t=>e.add(+t||0):i=>e.add(+n(i,++r,t)||0))},t.filter=function(t,n){if("function"!=typeof n)throw new TypeError("test is not a function");const e=[];let r=-1;for(const i of t)n(i,++r,t)&&e.push(i);return e},t.flatGroup=function(t,...n){return z(P(t,...n),n)},t.flatRollup=function(t,n,...e){return z(D(t,n,...e),e)},t.forceCenter=function(t,n){var e,r=1;function i(){var i,o,a=e.length,u=0,c=0;for(i=0;if+p||os+p||ac.index){var g=f-u.x-u.vx,y=s-u.y-u.vy,v=g*g+y*y;vt.r&&(t.r=t[n].r)}function c(){if(n){var r,i,o=n.length;for(e=new Array(o),r=0;r[u(t,n,r),t])));for(a=0,i=new Array(f);a=u)){(t.data!==n||t.next)&&(0===l&&(p+=(l=Uc(e))*l),0===h&&(p+=(h=Uc(e))*h),p(t=(Lc*t+jc)%Hc)/Hc}();function l(){h(),f.call("tick",n),e1?(null==e?u.delete(t):u.set(t,p(e)),n):u.get(t)},find:function(n,e,r){var i,o,a,u,c,f=0,s=t.length;for(null==r?r=1/0:r*=r,f=0;f1?(f.on(t,e),n):f.on(t)}}},t.forceX=function(t){var n,e,r,i=qc(.1);function o(t){for(var i,o=0,a=n.length;o=.12&&i<.234&&r>=-.425&&r<-.214?u:i>=.166&&i<.234&&r>=-.214&&r<-.115?c:a).invert(t)},s.stream=function(e){return t&&n===e?t:(r=[a.stream(n=e),u.stream(e),c.stream(e)],i=r.length,t={point:function(t,n){for(var e=-1;++ejs(r[0],r[1])&&(r[1]=i[1]),js(i[0],r[1])>js(r[0],r[1])&&(r[0]=i[0])):o.push(r=i);for(a=-1/0,n=0,r=o[e=o.length-1];n<=e;r=i,++n)i=o[n],(u=js(r[1],i[0]))>a&&(a=u,Wf=i[0],Kf=r[1])}return is=os=null,Wf===1/0||Zf===1/0?[[NaN,NaN],[NaN,NaN]]:[[Wf,Zf],[Kf,Qf]]},t.geoCentroid=function(t){ms=xs=ws=Ms=Ts=As=Ss=Es=0,Ns=new T,ks=new T,Cs=new T,Lf(t,Gs);var n=+Ns,e=+ks,r=+Cs,i=Ef(n,e,r);return i=0))throw new RangeError(`invalid digits: ${t}`);i=n}return null===n&&(r=new ed(i)),a},a.projection(t).digits(i).context(n)},t.geoProjection=yd,t.geoProjectionMutator=vd,t.geoRotation=ll,t.geoStereographic=function(){return yd(Bd).scale(250).clipAngle(142)},t.geoStereographicRaw=Bd,t.geoStream=Lf,t.geoTransform=function(t){return{stream:id(t)}},t.geoTransverseMercator=function(){var t=Ed(Yd),n=t.center,e=t.rotate;return t.center=function(t){return arguments.length?n([-t[1],t[0]]):[(t=n())[1],-t[0]]},t.rotate=function(t){return arguments.length?e([t[0],t[1],t.length>2?t[2]+90:90]):[(t=e())[0],t[1],t[2]-90]},e([0,0,90]).scale(159.155)},t.geoTransverseMercatorRaw=Yd,t.gray=function(t,n){return new ur(t,0,0,null==n?1:n)},t.greatest=ot,t.greatestIndex=function(t,e=n){if(1===e.length)return tt(t,e);let r,i=-1,o=-1;for(const n of t)++o,(i<0?0===e(n,n):e(n,r)>0)&&(r=n,i=o);return i},t.group=C,t.groupSort=function(t,e,r){return(2!==e.length?U($(t,e,r),(([t,e],[r,i])=>n(e,i)||n(t,r))):U(C(t,r),(([t,r],[i,o])=>e(r,o)||n(t,i)))).map((([t])=>t))},t.groups=P,t.hcl=dr,t.hierarchy=Gd,t.histogram=Q,t.hsl=He,t.html=Ec,t.image=function(t,n){return new Promise((function(e,r){var i=new Image;for(var o in n)i[o]=n[o];i.onerror=r,i.onload=function(){e(i)},i.src=t}))},t.index=function(t,...n){return F(t,k,R,n)},t.indexes=function(t,...n){return F(t,Array.from,R,n)},t.interpolate=Gr,t.interpolateArray=function(t,n){return(Ir(n)?Ur:Or)(t,n)},t.interpolateBasis=Er,t.interpolateBasisClosed=Nr,t.interpolateBlues=Gb,t.interpolateBrBG=ob,t.interpolateBuGn=Mb,t.interpolateBuPu=Ab,t.interpolateCividis=function(t){return t=Math.max(0,Math.min(1,t)),"rgb("+Math.max(0,Math.min(255,Math.round(-4.54-t*(35.34-t*(2381.73-t*(6402.7-t*(7024.72-2710.57*t)))))))+", "+Math.max(0,Math.min(255,Math.round(32.49+t*(170.73+t*(52.82-t*(131.46-t*(176.58-67.37*t)))))))+", "+Math.max(0,Math.min(255,Math.round(81.24+t*(442.36-t*(2482.43-t*(6167.24-t*(6614.94-2475.67*t)))))))+")"},t.interpolateCool=am,t.interpolateCubehelix=li,t.interpolateCubehelixDefault=im,t.interpolateCubehelixLong=hi,t.interpolateDate=Br,t.interpolateDiscrete=function(t){var n=t.length;return function(e){return t[Math.max(0,Math.min(n-1,Math.floor(e*n)))]}},t.interpolateGnBu=Eb,t.interpolateGreens=Wb,t.interpolateGreys=Kb,t.interpolateHcl=ci,t.interpolateHclLong=fi,t.interpolateHsl=oi,t.interpolateHslLong=ai,t.interpolateHue=function(t,n){var e=Pr(+t,+n);return function(t){var n=e(t);return n-360*Math.floor(n/360)}},t.interpolateInferno=pm,t.interpolateLab=function(t,n){var e=$r((t=ar(t)).l,(n=ar(n)).l),r=$r(t.a,n.a),i=$r(t.b,n.b),o=$r(t.opacity,n.opacity);return function(n){return t.l=e(n),t.a=r(n),t.b=i(n),t.opacity=o(n),t+""}},t.interpolateMagma=dm,t.interpolateNumber=Yr,t.interpolateNumberArray=Ur,t.interpolateObject=Lr,t.interpolateOrRd=kb,t.interpolateOranges=rm,t.interpolatePRGn=ub,t.interpolatePiYG=fb,t.interpolatePlasma=gm,t.interpolatePuBu=$b,t.interpolatePuBuGn=Pb,t.interpolatePuOr=lb,t.interpolatePuRd=Rb,t.interpolatePurples=Jb,t.interpolateRainbow=function(t){(t<0||t>1)&&(t-=Math.floor(t));var n=Math.abs(t-.5);return um.h=360*t-100,um.s=1.5-1.5*n,um.l=.8-.9*n,um+""},t.interpolateRdBu=db,t.interpolateRdGy=gb,t.interpolateRdPu=qb,t.interpolateRdYlBu=vb,t.interpolateRdYlGn=bb,t.interpolateReds=nm,t.interpolateRgb=Dr,t.interpolateRgbBasis=Fr,t.interpolateRgbBasisClosed=qr,t.interpolateRound=Vr,t.interpolateSinebow=function(t){var n;return t=(.5-t)*Math.PI,cm.r=255*(n=Math.sin(t))*n,cm.g=255*(n=Math.sin(t+fm))*n,cm.b=255*(n=Math.sin(t+sm))*n,cm+""},t.interpolateSpectral=xb,t.interpolateString=Xr,t.interpolateTransformCss=ti,t.interpolateTransformSvg=ni,t.interpolateTurbo=function(t){return t=Math.max(0,Math.min(1,t)),"rgb("+Math.max(0,Math.min(255,Math.round(34.61+t*(1172.33-t*(10793.56-t*(33300.12-t*(38394.49-14825.05*t)))))))+", "+Math.max(0,Math.min(255,Math.round(23.31+t*(557.33+t*(1225.33-t*(3574.96-t*(1073.77+707.56*t)))))))+", "+Math.max(0,Math.min(255,Math.round(27.2+t*(3211.1-t*(15327.97-t*(27814-t*(22569.18-6838.66*t)))))))+")"},t.interpolateViridis=hm,t.interpolateWarm=om,t.interpolateYlGn=Bb,t.interpolateYlGnBu=Ib,t.interpolateYlOrBr=Lb,t.interpolateYlOrRd=Hb,t.interpolateZoom=ri,t.interrupt=Gi,t.intersection=function(t,...n){t=new InternSet(t),n=n.map(vt);t:for(const e of t)for(const r of n)if(!r.has(e)){t.delete(e);continue t}return t},t.interval=function(t,n,e){var r=new Ei,i=n;return null==n?(r.restart(t,n,e),r):(r._restart=r.restart,r.restart=function(t,n,e){n=+n,e=null==e?Ai():+e,r._restart((function o(a){a+=i,r._restart(o,i+=n,e),t(a)}),n,e)},r.restart(t,n,e),r)},t.isoFormat=D_,t.isoParse=F_,t.json=function(t,n){return fetch(t,n).then(Tc)},t.lab=ar,t.lch=function(t,n,e,r){return 1===arguments.length?hr(t):new pr(e,n,t,null==r?1:r)},t.least=function(t,e=n){let r,i=!1;if(1===e.length){let o;for(const a of t){const t=e(a);(i?n(t,o)<0:0===n(t,t))&&(r=a,o=t,i=!0)}}else for(const n of t)(i?e(n,r)<0:0===e(n,n))&&(r=n,i=!0);return r},t.leastIndex=ht,t.line=Ym,t.lineRadial=Zm,t.link=ax,t.linkHorizontal=function(){return ax(nx)},t.linkRadial=function(){const t=ax(rx);return t.angle=t.x,delete t.x,t.radius=t.y,delete t.y,t},t.linkVertical=function(){return ax(ex)},t.local=Qn,t.map=function(t,n){if("function"!=typeof t[Symbol.iterator])throw new TypeError("values is not iterable");if("function"!=typeof n)throw new TypeError("mapper is not a function");return Array.from(t,((e,r)=>n(e,r,t)))},t.matcher=Vt,t.max=J,t.maxIndex=tt,t.mean=function(t,n){let e=0,r=0;if(void 0===n)for(let n of t)null!=n&&(n=+n)>=n&&(++e,r+=n);else{let i=-1;for(let o of t)null!=(o=n(o,++i,t))&&(o=+o)>=o&&(++e,r+=o)}if(e)return r/e},t.median=function(t,n){return at(t,.5,n)},t.medianIndex=function(t,n){return ct(t,.5,n)},t.merge=ft,t.min=nt,t.minIndex=et,t.mode=function(t,n){const e=new InternMap;if(void 0===n)for(let n of t)null!=n&&n>=n&&e.set(n,(e.get(n)||0)+1);else{let r=-1;for(let i of t)null!=(i=n(i,++r,t))&&i>=i&&e.set(i,(e.get(i)||0)+1)}let r,i=0;for(const[t,n]of e)n>i&&(i=n,r=t);return r},t.namespace=It,t.namespaces=Ut,t.nice=Z,t.now=Ai,t.pack=function(){var t=null,n=1,e=1,r=np;function i(i){const o=ap();return i.x=n/2,i.y=e/2,t?i.eachBefore(xp(t)).eachAfter(wp(r,.5,o)).eachBefore(Mp(1)):i.eachBefore(xp(mp)).eachAfter(wp(np,1,o)).eachAfter(wp(r,i.r/Math.min(n,e),o)).eachBefore(Mp(Math.min(n,e)/(2*i.r))),i}return i.radius=function(n){return arguments.length?(t=Jd(n),i):t},i.size=function(t){return arguments.length?(n=+t[0],e=+t[1],i):[n,e]},i.padding=function(t){return arguments.length?(r="function"==typeof t?t:ep(+t),i):r},i},t.packEnclose=function(t){return up(t,ap())},t.packSiblings=function(t){return bp(t,ap()),t},t.pairs=function(t,n=st){const e=[];let r,i=!1;for(const o of t)i&&e.push(n(r,o)),r=o,i=!0;return e},t.partition=function(){var t=1,n=1,e=0,r=!1;function i(i){var o=i.height+1;return i.x0=i.y0=e,i.x1=t,i.y1=n/o,i.eachBefore(function(t,n){return function(r){r.children&&Ap(r,r.x0,t*(r.depth+1)/n,r.x1,t*(r.depth+2)/n);var i=r.x0,o=r.y0,a=r.x1-e,u=r.y1-e;a0&&(d+=l);for(null!=n?p.sort((function(t,e){return n(g[t],g[e])})):null!=e&&p.sort((function(t,n){return e(a[t],a[n])})),u=0,f=d?(v-h*b)/d:0;u0?l*f:0)+b,g[c]={data:a[c],index:u,value:l,startAngle:y,endAngle:s,padAngle:_};return g}return a.value=function(n){return arguments.length?(t="function"==typeof n?n:ym(+n),a):t},a.sortValues=function(t){return arguments.length?(n=t,e=null,a):n},a.sort=function(t){return arguments.length?(e=t,n=null,a):e},a.startAngle=function(t){return arguments.length?(r="function"==typeof t?t:ym(+t),a):r},a.endAngle=function(t){return arguments.length?(i="function"==typeof t?t:ym(+t),a):i},a.padAngle=function(t){return arguments.length?(o="function"==typeof t?t:ym(+t),a):o},a},t.piecewise=di,t.pointRadial=Qm,t.pointer=ne,t.pointers=function(t,n){return t.target&&(t=te(t),void 0===n&&(n=t.currentTarget),t=t.touches||[t]),Array.from(t,(t=>ne(t,n)))},t.polygonArea=function(t){for(var n,e=-1,r=t.length,i=t[r-1],o=0;++eu!=f>u&&a<(c-e)*(u-r)/(f-r)+e&&(s=!s),c=e,f=r;return s},t.polygonHull=function(t){if((e=t.length)<3)return null;var n,e,r=new Array(e),i=new Array(e);for(n=0;n=0;--n)f.push(t[r[o[n]][2]]);for(n=+u;n(n=1664525*n+1013904223|0,lg*(n>>>0))},t.randomLogNormal=Kp,t.randomLogistic=fg,t.randomNormal=Zp,t.randomPareto=ng,t.randomPoisson=sg,t.randomUniform=Vp,t.randomWeibull=ug,t.range=lt,t.rank=function(t,e=n){if("function"!=typeof t[Symbol.iterator])throw new TypeError("values is not iterable");let r=Array.from(t);const i=new Float64Array(r.length);2!==e.length&&(r=r.map(e),e=n);const o=(t,n)=>e(r[t],r[n]);let a,u;return(t=Uint32Array.from(r,((t,n)=>n))).sort(e===n?(t,n)=>O(r[t],r[n]):I(o)),t.forEach(((t,n)=>{const e=o(t,void 0===a?t:a);e>=0?((void 0===a||e>0)&&(a=t,u=n),i[t]=u):i[t]=NaN})),i},t.reduce=function(t,n,e){if("function"!=typeof n)throw new TypeError("reducer is not a function");const r=t[Symbol.iterator]();let i,o,a=-1;if(arguments.length<3){if(({done:i,value:e}=r.next()),i)return;++a}for(;({done:i,value:o}=r.next()),!i;)e=n(e,o,++a,t);return e},t.reverse=function(t){if("function"!=typeof t[Symbol.iterator])throw new TypeError("values is not iterable");return Array.from(t).reverse()},t.rgb=Fe,t.ribbon=function(){return Wa()},t.ribbonArrow=function(){return Wa(Va)},t.rollup=$,t.rollups=D,t.scaleBand=yg,t.scaleDiverging=function t(){var n=Ng(L_()(mg));return n.copy=function(){return B_(n,t())},dg.apply(n,arguments)},t.scaleDivergingLog=function t(){var n=Fg(L_()).domain([.1,1,10]);return n.copy=function(){return B_(n,t()).base(n.base())},dg.apply(n,arguments)},t.scaleDivergingPow=j_,t.scaleDivergingSqrt=function(){return j_.apply(null,arguments).exponent(.5)},t.scaleDivergingSymlog=function t(){var n=Ig(L_());return n.copy=function(){return B_(n,t()).constant(n.constant())},dg.apply(n,arguments)},t.scaleIdentity=function t(n){var e;function r(t){return null==t||isNaN(t=+t)?e:t}return r.invert=r,r.domain=r.range=function(t){return arguments.length?(n=Array.from(t,_g),r):n.slice()},r.unknown=function(t){return arguments.length?(e=t,r):e},r.copy=function(){return t(n).unknown(e)},n=arguments.length?Array.from(n,_g):[0,1],Ng(r)},t.scaleImplicit=pg,t.scaleLinear=function t(){var n=Sg();return n.copy=function(){return Tg(n,t())},hg.apply(n,arguments),Ng(n)},t.scaleLog=function t(){const n=Fg(Ag()).domain([1,10]);return n.copy=()=>Tg(n,t()).base(n.base()),hg.apply(n,arguments),n},t.scaleOrdinal=gg,t.scalePoint=function(){return vg(yg.apply(null,arguments).paddingInner(1))},t.scalePow=jg,t.scaleQuantile=function t(){var e,r=[],i=[],o=[];function a(){var t=0,n=Math.max(1,i.length);for(o=new Array(n-1);++t0?o[n-1]:r[0],n=i?[o[i-1],r]:[o[n-1],o[n]]},u.unknown=function(t){return arguments.length?(n=t,u):u},u.thresholds=function(){return o.slice()},u.copy=function(){return t().domain([e,r]).range(a).unknown(n)},hg.apply(Ng(u),arguments)},t.scaleRadial=function t(){var n,e=Sg(),r=[0,1],i=!1;function o(t){var r=function(t){return Math.sign(t)*Math.sqrt(Math.abs(t))}(e(t));return isNaN(r)?n:i?Math.round(r):r}return o.invert=function(t){return e.invert(Hg(t))},o.domain=function(t){return arguments.length?(e.domain(t),o):e.domain()},o.range=function(t){return arguments.length?(e.range((r=Array.from(t,_g)).map(Hg)),o):r.slice()},o.rangeRound=function(t){return o.range(t).round(!0)},o.round=function(t){return arguments.length?(i=!!t,o):i},o.clamp=function(t){return arguments.length?(e.clamp(t),o):e.clamp()},o.unknown=function(t){return arguments.length?(n=t,o):n},o.copy=function(){return t(e.domain(),r).round(i).clamp(e.clamp()).unknown(n)},hg.apply(o,arguments),Ng(o)},t.scaleSequential=function t(){var n=Ng(O_()(mg));return n.copy=function(){return B_(n,t())},dg.apply(n,arguments)},t.scaleSequentialLog=function t(){var n=Fg(O_()).domain([1,10]);return n.copy=function(){return B_(n,t()).base(n.base())},dg.apply(n,arguments)},t.scaleSequentialPow=Y_,t.scaleSequentialQuantile=function t(){var e=[],r=mg;function i(t){if(null!=t&&!isNaN(t=+t))return r((s(e,t,1)-1)/(e.length-1))}return i.domain=function(t){if(!arguments.length)return e.slice();e=[];for(let n of t)null==n||isNaN(n=+n)||e.push(n);return e.sort(n),i},i.interpolator=function(t){return arguments.length?(r=t,i):r},i.range=function(){return e.map(((t,n)=>r(n/(e.length-1))))},i.quantiles=function(t){return Array.from({length:t+1},((n,r)=>at(e,r/t)))},i.copy=function(){return t(r).domain(e)},dg.apply(i,arguments)},t.scaleSequentialSqrt=function(){return Y_.apply(null,arguments).exponent(.5)},t.scaleSequentialSymlog=function t(){var n=Ig(O_());return n.copy=function(){return B_(n,t()).constant(n.constant())},dg.apply(n,arguments)},t.scaleSqrt=function(){return jg.apply(null,arguments).exponent(.5)},t.scaleSymlog=function t(){var n=Ig(Ag());return n.copy=function(){return Tg(n,t()).constant(n.constant())},hg.apply(n,arguments)},t.scaleThreshold=function t(){var n,e=[.5],r=[0,1],i=1;function o(t){return null!=t&&t<=t?r[s(e,t,0,i)]:n}return o.domain=function(t){return arguments.length?(e=Array.from(t),i=Math.min(e.length,r.length-1),o):e.slice()},o.range=function(t){return arguments.length?(r=Array.from(t),i=Math.min(e.length,r.length-1),o):r.slice()},o.invertExtent=function(t){var n=r.indexOf(t);return[e[n-1],e[n]]},o.unknown=function(t){return arguments.length?(n=t,o):n},o.copy=function(){return t().domain(e).range(r).unknown(n)},hg.apply(o,arguments)},t.scaleTime=function(){return hg.apply(I_(uv,cv,tv,Zy,xy,py,sy,ay,iy,t.timeFormat).domain([new Date(2e3,0,1),new Date(2e3,0,2)]),arguments)},t.scaleUtc=function(){return hg.apply(I_(ov,av,ev,Qy,Fy,yy,hy,cy,iy,t.utcFormat).domain([Date.UTC(2e3,0,1),Date.UTC(2e3,0,2)]),arguments)},t.scan=function(t,n){const e=ht(t,n);return e<0?void 0:e},t.schemeAccent=G_,t.schemeBlues=Xb,t.schemeBrBG=ib,t.schemeBuGn=wb,t.schemeBuPu=Tb,t.schemeCategory10=X_,t.schemeDark2=V_,t.schemeGnBu=Sb,t.schemeGreens=Vb,t.schemeGreys=Zb,t.schemeObservable10=W_,t.schemeOrRd=Nb,t.schemeOranges=em,t.schemePRGn=ab,t.schemePaired=Z_,t.schemePastel1=K_,t.schemePastel2=Q_,t.schemePiYG=cb,t.schemePuBu=zb,t.schemePuBuGn=Cb,t.schemePuOr=sb,t.schemePuRd=Db,t.schemePurples=Qb,t.schemeRdBu=hb,t.schemeRdGy=pb,t.schemeRdPu=Fb,t.schemeRdYlBu=yb,t.schemeRdYlGn=_b,t.schemeReds=tm,t.schemeSet1=J_,t.schemeSet2=tb,t.schemeSet3=nb,t.schemeSpectral=mb,t.schemeTableau10=eb,t.schemeYlGn=Ob,t.schemeYlGnBu=Ub,t.schemeYlOrBr=Yb,t.schemeYlOrRd=jb,t.select=Zn,t.selectAll=function(t){return"string"==typeof t?new Vn([document.querySelectorAll(t)],[document.documentElement]):new Vn([Ht(t)],Gn)},t.selection=Wn,t.selector=jt,t.selectorAll=Gt,t.shuffle=dt,t.shuffler=pt,t.some=function(t,n){if("function"!=typeof n)throw new TypeError("test is not a function");let e=-1;for(const r of t)if(n(r,++e,t))return!0;return!1},t.sort=U,t.stack=function(){var t=ym([]),n=dw,e=hw,r=pw;function i(i){var o,a,u=Array.from(t.apply(this,arguments),gw),c=u.length,f=-1;for(const t of i)for(o=0,++f;o0)for(var e,r,i,o,a,u,c=0,f=t[n[0]].length;c0?(r[0]=o,r[1]=o+=i):i<0?(r[1]=a,r[0]=a+=i):(r[0]=0,r[1]=i)},t.stackOffsetExpand=function(t,n){if((r=t.length)>0){for(var e,r,i,o=0,a=t[0].length;o0){for(var e,r=0,i=t[n[0]],o=i.length;r0&&(r=(e=t[n[0]]).length)>0){for(var e,r,i,o=0,a=1;afunction(t){t=`${t}`;let n=t.length;zp(t,n-1)&&!zp(t,n-2)&&(t=t.slice(0,-1));return"/"===t[0]?t:`/${t}`}(t(n,e,r)))),e=n.map(Pp),i=new Set(n).add("");for(const t of e)i.has(t)||(i.add(t),n.push(t),e.push(Pp(t)),h.push(Np));d=(t,e)=>n[e],p=(t,n)=>e[n]}for(a=0,i=h.length;a=0&&(f=h[t]).data===Np;--t)f.data=null}if(u.parent=Sp,u.eachBefore((function(t){t.depth=t.parent.depth+1,--i})).eachBefore(Kd),u.parent=null,i>0)throw new Error("cycle");return u}return r.id=function(t){return arguments.length?(n=Jd(t),r):n},r.parentId=function(t){return arguments.length?(e=Jd(t),r):e},r.path=function(n){return arguments.length?(t=Jd(n),r):t},r},t.style=_n,t.subset=function(t,n){return _t(n,t)},t.sum=function(t,n){let e=0;if(void 0===n)for(let n of t)(n=+n)&&(e+=n);else{let r=-1;for(let i of t)(i=+n(i,++r,t))&&(e+=i)}return e},t.superset=_t,t.svg=Nc,t.symbol=function(t,n){let e=null,r=km(i);function i(){let i;if(e||(e=i=r()),t.apply(this,arguments).draw(e,+n.apply(this,arguments)),i)return e=null,i+""||null}return t="function"==typeof t?t:ym(t||fx),n="function"==typeof n?n:ym(void 0===n?64:+n),i.type=function(n){return arguments.length?(t="function"==typeof n?n:ym(n),i):t},i.size=function(t){return arguments.length?(n="function"==typeof t?t:ym(+t),i):n},i.context=function(t){return arguments.length?(e=null==t?null:t,i):e},i},t.symbolAsterisk=cx,t.symbolCircle=fx,t.symbolCross=sx,t.symbolDiamond=dx,t.symbolDiamond2=px,t.symbolPlus=gx,t.symbolSquare=yx,t.symbolSquare2=vx,t.symbolStar=xx,t.symbolTimes=Px,t.symbolTriangle=Mx,t.symbolTriangle2=Ax,t.symbolWye=Cx,t.symbolX=Px,t.symbols=zx,t.symbolsFill=zx,t.symbolsStroke=$x,t.text=mc,t.thresholdFreedmanDiaconis=function(t,n,e){const r=v(t),i=at(t,.75)-at(t,.25);return r&&i?Math.ceil((e-n)/(2*i*Math.pow(r,-1/3))):1},t.thresholdScott=function(t,n,e){const r=v(t),i=w(t);return r&&i?Math.ceil((e-n)*Math.cbrt(r)/(3.49*i)):1},t.thresholdSturges=K,t.tickFormat=Eg,t.tickIncrement=V,t.tickStep=W,t.ticks=G,t.timeDay=py,t.timeDays=gy,t.timeFormatDefaultLocale=P_,t.timeFormatLocale=hv,t.timeFriday=Sy,t.timeFridays=$y,t.timeHour=sy,t.timeHours=ly,t.timeInterval=Vg,t.timeMillisecond=Wg,t.timeMilliseconds=Zg,t.timeMinute=ay,t.timeMinutes=uy,t.timeMonday=wy,t.timeMondays=ky,t.timeMonth=Zy,t.timeMonths=Ky,t.timeSaturday=Ey,t.timeSaturdays=Dy,t.timeSecond=iy,t.timeSeconds=oy,t.timeSunday=xy,t.timeSundays=Ny,t.timeThursday=Ay,t.timeThursdays=zy,t.timeTickInterval=cv,t.timeTicks=uv,t.timeTuesday=My,t.timeTuesdays=Cy,t.timeWednesday=Ty,t.timeWednesdays=Py,t.timeWeek=xy,t.timeWeeks=Ny,t.timeYear=tv,t.timeYears=nv,t.timeout=$i,t.timer=Ni,t.timerFlush=ki,t.transition=go,t.transpose=gt,t.tree=function(){var t=$p,n=1,e=1,r=null;function i(i){var c=function(t){for(var n,e,r,i,o,a=new Up(t,0),u=[a];n=u.pop();)if(r=n._.children)for(n.children=new Array(o=r.length),i=o-1;i>=0;--i)u.push(e=n.children[i]=new Up(r[i],i)),e.parent=n;return(a.parent=new Up(null,0)).children=[a],a}(i);if(c.eachAfter(o),c.parent.m=-c.z,c.eachBefore(a),r)i.eachBefore(u);else{var f=i,s=i,l=i;i.eachBefore((function(t){t.xs.x&&(s=t),t.depth>l.depth&&(l=t)}));var h=f===s?1:t(f,s)/2,d=h-f.x,p=n/(s.x+h+d),g=e/(l.depth||1);i.eachBefore((function(t){t.x=(t.x+d)*p,t.y=t.depth*g}))}return i}function o(n){var e=n.children,r=n.parent.children,i=n.i?r[n.i-1]:null;if(e){!function(t){for(var n,e=0,r=0,i=t.children,o=i.length;--o>=0;)(n=i[o]).z+=e,n.m+=e,e+=n.s+(r+=n.c)}(n);var o=(e[0].z+e[e.length-1].z)/2;i?(n.z=i.z+t(n._,i._),n.m=n.z-o):n.z=o}else i&&(n.z=i.z+t(n._,i._));n.parent.A=function(n,e,r){if(e){for(var i,o=n,a=n,u=e,c=o.parent.children[0],f=o.m,s=a.m,l=u.m,h=c.m;u=Rp(u),o=Dp(o),u&&o;)c=Dp(c),(a=Rp(a)).a=n,(i=u.z+l-o.z-f+t(u._,o._))>0&&(Fp(qp(u,n,r),n,i),f+=i,s+=i),l+=u.m,f+=o.m,h+=c.m,s+=a.m;u&&!Rp(a)&&(a.t=u,a.m+=l-s),o&&!Dp(c)&&(c.t=o,c.m+=f-h,r=n)}return r}(n,i,n.parent.A||r[0])}function a(t){t._.x=t.z+t.parent.m,t.m+=t.parent.m}function u(t){t.x*=n,t.y=t.depth*e}return i.separation=function(n){return arguments.length?(t=n,i):t},i.size=function(t){return arguments.length?(r=!1,n=+t[0],e=+t[1],i):r?null:[n,e]},i.nodeSize=function(t){return arguments.length?(r=!0,n=+t[0],e=+t[1],i):r?[n,e]:null},i},t.treemap=function(){var t=Yp,n=!1,e=1,r=1,i=[0],o=np,a=np,u=np,c=np,f=np;function s(t){return t.x0=t.y0=0,t.x1=e,t.y1=r,t.eachBefore(l),i=[0],n&&t.eachBefore(Tp),t}function l(n){var e=i[n.depth],r=n.x0+e,s=n.y0+e,l=n.x1-e,h=n.y1-e;l=e-1){var s=u[n];return s.x0=i,s.y0=o,s.x1=a,void(s.y1=c)}var l=f[n],h=r/2+l,d=n+1,p=e-1;for(;d>>1;f[g]c-o){var _=r?(i*v+a*y)/r:a;t(n,d,y,i,o,_,c),t(d,e,v,_,o,a,c)}else{var b=r?(o*v+c*y)/r:c;t(n,d,y,i,o,a,b),t(d,e,v,i,b,a,c)}}(0,c,t.value,n,e,r,i)},t.treemapDice=Ap,t.treemapResquarify=Lp,t.treemapSlice=Ip,t.treemapSliceDice=function(t,n,e,r,i){(1&t.depth?Ip:Ap)(t,n,e,r,i)},t.treemapSquarify=Yp,t.tsv=Mc,t.tsvFormat=lc,t.tsvFormatBody=hc,t.tsvFormatRow=pc,t.tsvFormatRows=dc,t.tsvFormatValue=gc,t.tsvParse=fc,t.tsvParseRows=sc,t.union=function(...t){const n=new InternSet;for(const e of t)for(const t of e)n.add(t);return n},t.unixDay=_y,t.unixDays=by,t.utcDay=yy,t.utcDays=vy,t.utcFriday=By,t.utcFridays=Vy,t.utcHour=hy,t.utcHours=dy,t.utcMillisecond=Wg,t.utcMilliseconds=Zg,t.utcMinute=cy,t.utcMinutes=fy,t.utcMonday=qy,t.utcMondays=jy,t.utcMonth=Qy,t.utcMonths=Jy,t.utcSaturday=Yy,t.utcSaturdays=Wy,t.utcSecond=iy,t.utcSeconds=oy,t.utcSunday=Fy,t.utcSundays=Ly,t.utcThursday=Oy,t.utcThursdays=Gy,t.utcTickInterval=av,t.utcTicks=ov,t.utcTuesday=Uy,t.utcTuesdays=Hy,t.utcWednesday=Iy,t.utcWednesdays=Xy,t.utcWeek=Fy,t.utcWeeks=Ly,t.utcYear=ev,t.utcYears=rv,t.variance=x,t.version="7.9.0",t.window=pn,t.xml=Sc,t.zip=function(){return gt(arguments)},t.zoom=function(){var t,n,e,r=Ew,i=Nw,o=zw,a=Cw,u=Pw,c=[0,1/0],f=[[-1/0,-1/0],[1/0,1/0]],s=250,l=ri,h=$t("start","zoom","end"),d=500,p=150,g=0,y=10;function v(t){t.property("__zoom",kw).on("wheel.zoom",T,{passive:!1}).on("mousedown.zoom",A).on("dblclick.zoom",S).filter(u).on("touchstart.zoom",E).on("touchmove.zoom",N).on("touchend.zoom touchcancel.zoom",k).style("-webkit-tap-highlight-color","rgba(0,0,0,0)")}function _(t,n){return(n=Math.max(c[0],Math.min(c[1],n)))===t.k?t:new ww(n,t.x,t.y)}function b(t,n,e){var r=n[0]-e[0]*t.k,i=n[1]-e[1]*t.k;return r===t.x&&i===t.y?t:new ww(t.k,r,i)}function m(t){return[(+t[0][0]+ +t[1][0])/2,(+t[0][1]+ +t[1][1])/2]}function x(t,n,e,r){t.on("start.zoom",(function(){w(this,arguments).event(r).start()})).on("interrupt.zoom end.zoom",(function(){w(this,arguments).event(r).end()})).tween("zoom",(function(){var t=this,o=arguments,a=w(t,o).event(r),u=i.apply(t,o),c=null==e?m(u):"function"==typeof e?e.apply(t,o):e,f=Math.max(u[1][0]-u[0][0],u[1][1]-u[0][1]),s=t.__zoom,h="function"==typeof n?n.apply(t,o):n,d=l(s.invert(c).concat(f/s.k),h.invert(c).concat(f/h.k));return function(t){if(1===t)t=h;else{var n=d(t),e=f/n[2];t=new ww(e,c[0]-n[0]*e,c[1]-n[1]*e)}a.zoom(null,t)}}))}function w(t,n,e){return!e&&t.__zooming||new M(t,n)}function M(t,n){this.that=t,this.args=n,this.active=0,this.sourceEvent=null,this.extent=i.apply(t,n),this.taps=0}function T(t,...n){if(r.apply(this,arguments)){var e=w(this,n).event(t),i=this.__zoom,u=Math.max(c[0],Math.min(c[1],i.k*Math.pow(2,a.apply(this,arguments)))),s=ne(t);if(e.wheel)e.mouse[0][0]===s[0]&&e.mouse[0][1]===s[1]||(e.mouse[1]=i.invert(e.mouse[0]=s)),clearTimeout(e.wheel);else{if(i.k===u)return;e.mouse=[s,i.invert(s)],Gi(this),e.start()}Sw(t),e.wheel=setTimeout((function(){e.wheel=null,e.end()}),p),e.zoom("mouse",o(b(_(i,u),e.mouse[0],e.mouse[1]),e.extent,f))}}function A(t,...n){if(!e&&r.apply(this,arguments)){var i=t.currentTarget,a=w(this,n,!0).event(t),u=Zn(t.view).on("mousemove.zoom",(function(t){if(Sw(t),!a.moved){var n=t.clientX-s,e=t.clientY-l;a.moved=n*n+e*e>g}a.event(t).zoom("mouse",o(b(a.that.__zoom,a.mouse[0]=ne(t,i),a.mouse[1]),a.extent,f))}),!0).on("mouseup.zoom",(function(t){u.on("mousemove.zoom mouseup.zoom",null),ue(t.view,a.moved),Sw(t),a.event(t).end()}),!0),c=ne(t,i),s=t.clientX,l=t.clientY;ae(t.view),Aw(t),a.mouse=[c,this.__zoom.invert(c)],Gi(this),a.start()}}function S(t,...n){if(r.apply(this,arguments)){var e=this.__zoom,a=ne(t.changedTouches?t.changedTouches[0]:t,this),u=e.invert(a),c=e.k*(t.shiftKey?.5:2),l=o(b(_(e,c),a,u),i.apply(this,n),f);Sw(t),s>0?Zn(this).transition().duration(s).call(x,l,a,t):Zn(this).call(v.transform,l,a,t)}}function E(e,...i){if(r.apply(this,arguments)){var o,a,u,c,f=e.touches,s=f.length,l=w(this,i,e.changedTouches.length===s).event(e);for(Aw(e),a=0;a { + proj = createTempProject(FIXTURE_DIR); + server = await startServer(proj.dir); +}); + +afterAll(() => { + server?.stop(); + proj?.cleanup(); +}); + +function readValidatorEmits(projectDir) { + const sessionsDir = join(projectDir, '.pos-supervisor', 'sessions'); + if (!existsSync(sessionsDir)) return []; + const entries = readdirSync(sessionsDir, { withFileTypes: true }).filter(e => e.isDirectory()); + const out = []; + for (const entry of entries) { + const eventsPath = join(sessionsDir, entry.name, 'events.ndjson'); + if (!existsSync(eventsPath)) continue; + for (const line of readFileSync(eventsPath, 'utf-8').split('\n').filter(Boolean)) { + try { + const ev = JSON.parse(line); + if (ev.kind === 'validator_emit') out.push(ev); + } catch { /* skip */ } + } + } + return out; +} + +describe('emit-loop propagates rule_id onto every fix (I1 follow-up)', () => { + // A page rendering a partial that doesn't exist triggers MissingPartial. + // In quick mode the rule engine's `MissingPartial.create_file` branch fires + // (priority 20 — nearest-match is priority 10, won't hit for a truly + // unknown name). Its HintResult carries rule_id AND fixes[0] is a + // create_file. The fix object itself has no rule_id — we're testing that + // the emit loop inherits d.rule_id. + it('MissingPartial.create_file fix inherits rule_id from the diagnostic', async () => { + const FILE = 'app/views/pages/rule-attr-test.liquid'; + const CONTENT = '---\nslug: rule-attr-test\n---\n{% render "totally/nonexistent_partial_xyz" %}\n'; + + await server.callTool('validate_code', { + file_path: FILE, + content: CONTENT, + mode: 'full', + }); + + await new Promise(r => setTimeout(r, 100)); + + const emits = readValidatorEmits(proj.dir); + const mpEmit = emits.find( + e => e.file === FILE && e.check === 'MissingPartial' && (e.proposed_fixes?.length ?? 0) > 0, + ); + expect(mpEmit).toBeDefined(); + expect(mpEmit.hint_rule_id).toContain('MissingPartial'); + + // Every fix attached to this diagnostic carries a rule_id — either its + // own (heuristic:...) or inherited from the rule engine's d.rule_id. + for (const fix of mpEmit.proposed_fixes) { + expect(fix.rule_id).not.toBeNull(); + expect(typeof fix.rule_id).toBe('string'); + } + + // At least one fix should be attributed to the rule-engine rule, not + // solely the heuristic generator — that's the regression guard. + const hasRuleEngineAttribution = mpEmit.proposed_fixes.some( + f => f.rule_id && !f.rule_id.startsWith('heuristic:'), + ); + expect(hasRuleEngineAttribution).toBe(true); + }); +}); diff --git a/tests/integration/analytics/force-disable-check.test.js b/tests/integration/analytics/force-disable-check.test.js new file mode 100644 index 0000000..bdd4f52 --- /dev/null +++ b/tests/integration/analytics/force-disable-check.test.js @@ -0,0 +1,87 @@ +/** + * I4 — force-disable works on check names, not just rule_ids. + * + * An operator looking at a HARMFUL row in Rule Performance wants a + * one-click "stop emitting this diagnostic". Today's force-disable only + * gates rule_ids inside `runRules()`; structural checks (pos-supervisor:*) + * and LSP checks without a rule module never hit that path. The fix adds + * a filter in validate-code.js that drops diagnostics whose check name or + * rule_id appears in the force-disable set. + * + * This test exercises the end-to-end path: write the override file → boot + * the server → run validate_code → assert the suppressed diagnostic is + * absent and clearing the override restores it. + */ + +import { describe, it, expect, beforeAll, afterAll, setDefaultTimeout } from 'bun:test'; +import { readFileSync } from 'node:fs'; +import { startServer, FIXTURE_DIR, createTempProject } from '../helpers/server.js'; +import { loadOverrides } from '../../../src/core/rule-overrides.js'; + +setDefaultTimeout(60_000); + +let server; +let proj; + +// A page with inline HTML and no partial renders → triggers +// pos-supervisor:HtmlInPage (our B-tier guard only suppresses when +// renders_used is non-empty, which this content isn't). +const FILE = 'app/views/pages/force-disable-test.liquid'; +const CONTENT = '---\nslug: force-disable-test\n---\n

hi

\n'; + +beforeAll(async () => { + proj = createTempProject(FIXTURE_DIR); + server = await startServer(proj.dir); +}); + +afterAll(() => { + server?.stop(); + proj?.cleanup(); +}); + +async function runValidate() { + return server.callTool('validate_code', { + file_path: FILE, + content: CONTENT, + mode: 'quick', + }); +} + +describe('force-disable on a check name', () => { + it('baseline: pos-supervisor:HtmlInPage fires on pure-HTML page', async () => { + const res = await runValidate(); + const all = [...(res.errors ?? []), ...(res.warnings ?? [])]; + expect(all.some(d => d.check === 'pos-supervisor:HtmlInPage')).toBe(true); + }); + + it('override suppresses the check; clearing it restores', async () => { + // Mirror the dashboard button path: POST to the endpoint, which writes + // the override file AND calls onOverridesChanged so the in-memory + // engine sees the new set without a restart. + const addResp = await fetch(server.baseUrl + '/api/engine/rule-overrides', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ action: 'force_disable', rule_id: 'pos-supervisor:HtmlInPage', reason: 'noisy on landing' }), + }); + const addBody = await addResp.json(); + expect(addResp.status).toBe(200); + expect(addBody.force_disable?.['pos-supervisor:HtmlInPage']).toBeDefined(); + // Sanity check that the file was written. + expect(loadOverrides(proj.dir).force_disable['pos-supervisor:HtmlInPage']).toBeDefined(); + + const suppressed = await runValidate(); + const suppressedAll = [...(suppressed.errors ?? []), ...(suppressed.warnings ?? [])]; + expect(suppressedAll.some(d => d.check === 'pos-supervisor:HtmlInPage')).toBe(false); + + // Clear and re-run — the diagnostic returns. + const clearResp = await fetch(server.baseUrl + '/api/engine/rule-overrides', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ action: 'clear', rule_id: 'pos-supervisor:HtmlInPage' }), + }); + expect(clearResp.ok).toBe(true); + await clearResp.json(); + + const restored = await runValidate(); + const restoredAll = [...(restored.errors ?? []), ...(restored.warnings ?? [])]; + expect(restoredAll.some(d => d.check === 'pos-supervisor:HtmlInPage')).toBe(true); + }); +}); diff --git a/tests/integration/analytics/structural-rule-attribution.test.js b/tests/integration/analytics/structural-rule-attribution.test.js new file mode 100644 index 0000000..17818f6 --- /dev/null +++ b/tests/integration/analytics/structural-rule-attribution.test.js @@ -0,0 +1,79 @@ +/** + * Structural-warning → rule-engine bridge (2026-04-24 fix). + * + * Before the bridge, structural checks with rule modules (like + * `pos-supervisor:NonGetRenderingPage`) landed in analytics as + * `.unmatched` because their rule modules never got invoked — + * enrichAll runs before structural warnings are pushed. The bridge calls + * runRules() a second time on any diagnostic still missing rule_id. + * + * This test drives a real validate_code call that emits a structural + * warning and asserts the resulting validator_emit carries the rule's + * canonical rule_id, not the `.unmatched` fallback. + */ + +import { describe, it, expect, beforeAll, afterAll, setDefaultTimeout } from 'bun:test'; +import { readFileSync, readdirSync, existsSync } from 'node:fs'; +import { join } from 'node:path'; +import { startServer, FIXTURE_DIR, createTempProject } from '../helpers/server.js'; + +setDefaultTimeout(30_000); + +let server; +let proj; + +beforeAll(async () => { + proj = createTempProject(FIXTURE_DIR); + server = await startServer(proj.dir); +}); + +afterAll(() => { + server?.stop(); + proj?.cleanup(); +}); + +function readValidatorEmits(projectDir) { + const sessionsDir = join(projectDir, '.pos-supervisor', 'sessions'); + if (!existsSync(sessionsDir)) return []; + const entries = readdirSync(sessionsDir, { withFileTypes: true }).filter(e => e.isDirectory()); + const out = []; + for (const entry of entries) { + const eventsPath = join(sessionsDir, entry.name, 'events.ndjson'); + if (!existsSync(eventsPath)) continue; + for (const line of readFileSync(eventsPath, 'utf-8').split('\n').filter(Boolean)) { + try { + const ev = JSON.parse(line); + if (ev.kind === 'validator_emit') out.push(ev); + } catch { /* skip */ } + } + } + return out; +} + +describe('structural rule attribution via bridge', () => { + // A page with method: post + HTML body — exactly the NonGetRenderingPage + // trigger. No slug under /api/, renders inline HTML and interpolates a + // variable so the UI-signal heuristics match. After task-4's three-subrule + // split (html_on_post / api_renders_html / get_form_target) this case + // routes to `html_on_post` rather than the catch-all `.default`. + it('NonGetRenderingPage lands as a specific subrule via the bridge, not ".unmatched"', async () => { + const FILE = 'app/views/pages/ngrp-bridge-test.liquid'; + const CONTENT = '---\nslug: ngrp-bridge-test\nmethod: post\nlayout: application\n---\n

form

\n{{ x }}\n'; + + await server.callTool('validate_code', { + file_path: FILE, + content: CONTENT, + mode: 'quick', + }); + + await new Promise(r => setTimeout(r, 100)); + + const emits = readValidatorEmits(proj.dir); + const ngrp = emits.find(e => e.file === FILE && e.check === 'pos-supervisor:NonGetRenderingPage'); + expect(ngrp).toBeDefined(); + // Non-API slug + HTML body + non-GET method → html_on_post subrule. + expect(ngrp.hint_rule_id).toBe('NonGetRenderingPage.html_on_post'); + // Confidence also propagates through the bridge. + expect(ngrp.confidence).toBe(0.9); + }); +}); diff --git a/tests/integration/analytics/untracked.test.js b/tests/integration/analytics/untracked.test.js new file mode 100644 index 0000000..2afe42c --- /dev/null +++ b/tests/integration/analytics/untracked.test.js @@ -0,0 +1,135 @@ +/** + * A3 — `_source: 'dashboard_live'` must not pollute analytics. + * + * The dashboard Live Diagnostic Console calls `validate_code` against + * experimental snippets. Those calls are interactive debugging, not agent + * activity — they must not write tool_call events, validator_emit events, or + * session-state mutations to the analytics pipeline. + */ + +import { describe, it, expect, beforeAll, afterAll, setDefaultTimeout } from 'bun:test'; +import { readFileSync, readdirSync, existsSync } from 'node:fs'; +import { join } from 'node:path'; +import { startServer, FIXTURE_DIR, createTempProject } from '../helpers/server.js'; + +setDefaultTimeout(30_000); + +let server; +let proj; + +beforeAll(async () => { + proj = createTempProject(FIXTURE_DIR); + server = await startServer(proj.dir); +}); + +afterAll(() => { + server?.stop(); + proj?.cleanup(); +}); + +function readSessionEvents(projectDir) { + const sessionsDir = join(projectDir, '.pos-supervisor', 'sessions'); + if (!existsSync(sessionsDir)) return []; + const entries = readdirSync(sessionsDir, { withFileTypes: true }) + .filter(e => e.isDirectory()); + const events = []; + for (const entry of entries) { + const eventsPath = join(sessionsDir, entry.name, 'events.ndjson'); + if (!existsSync(eventsPath)) continue; + const lines = readFileSync(eventsPath, 'utf-8').split('\n').filter(Boolean); + for (const line of lines) { + try { events.push(JSON.parse(line)); } catch { /* skip malformed */ } + } + } + return events; +} + +describe('_source: dashboard_live gates analytics writes (A3)', () => { + const SNIPPET_PATH = 'app/views/partials/__pos_live_console__.liquid'; + const SNIPPET_CONTENT = "{% assign foo = 'bar' %}\n

{{ foo }}

\n"; + + it('tracked validate_code writes a tool_call + validator_emit', async () => { + const before = readSessionEvents(proj.dir); + const beforeToolCalls = before.filter(e => e.kind === 'tool_call').length; + const beforeValidatorEmits = before.filter(e => e.kind === 'validator_emit').length; + + await server.callTool('validate_code', { + file_path: SNIPPET_PATH, + content: SNIPPET_CONTENT, + mode: 'quick', + }); + + // Give the writer a tick to flush. + await new Promise(r => setTimeout(r, 50)); + + const after = readSessionEvents(proj.dir); + const afterToolCalls = after.filter(e => e.kind === 'tool_call').length; + const afterValidatorEmits = after.filter(e => e.kind === 'validator_emit').length; + + expect(afterToolCalls).toBeGreaterThan(beforeToolCalls); + // UnusedAssign fires on the live-console snippet → at least one emit. + expect(afterValidatorEmits).toBeGreaterThan(beforeValidatorEmits); + }); + + it('untracked validate_code (dashboard_live) emits no tool_call or validator_emit', async () => { + const before = readSessionEvents(proj.dir); + const beforeToolCalls = before.filter(e => e.kind === 'tool_call').length; + const beforeValidatorEmits = before.filter(e => e.kind === 'validator_emit').length; + + await server.callTool('validate_code', { + file_path: SNIPPET_PATH, + content: SNIPPET_CONTENT, + mode: 'quick', + _source: 'dashboard_live', + }); + + await new Promise(r => setTimeout(r, 50)); + + const after = readSessionEvents(proj.dir); + const afterToolCalls = after.filter(e => e.kind === 'tool_call').length; + const afterValidatorEmits = after.filter(e => e.kind === 'validator_emit').length; + + expect(afterToolCalls).toBe(beforeToolCalls); + expect(afterValidatorEmits).toBe(beforeValidatorEmits); + }); + + it('untracked validate_code still returns diagnostics to the caller', async () => { + const result = await server.callTool('validate_code', { + file_path: SNIPPET_PATH, + content: SNIPPET_CONTENT, + mode: 'quick', + _source: 'dashboard_live', + }); + // The live console depends on this return path — untracked must not drop + // the response; only the analytics emission is gated. + expect(result).toBeDefined(); + expect(Array.isArray(result.errors)).toBe(true); + expect(Array.isArray(result.warnings)).toBe(true); + }); + + it('ctx.untracked is cleared after the call so subsequent tracked calls emit again', async () => { + // Run untracked first, then tracked — the second call must write events. + await server.callTool('validate_code', { + file_path: SNIPPET_PATH, + content: SNIPPET_CONTENT, + mode: 'quick', + _source: 'dashboard_live', + }); + + const before = readSessionEvents(proj.dir); + const beforeToolCalls = before.filter(e => e.kind === 'tool_call').length; + + await server.callTool('validate_code', { + file_path: SNIPPET_PATH, + content: SNIPPET_CONTENT, + mode: 'quick', + }); + + await new Promise(r => setTimeout(r, 50)); + + const after = readSessionEvents(proj.dir); + const afterToolCalls = after.filter(e => e.kind === 'tool_call').length; + + expect(afterToolCalls).toBeGreaterThan(beforeToolCalls); + }); +}); diff --git a/tests/integration/analyze-project-lib-prefix.integration.test.js b/tests/integration/analyze-project-lib-prefix.integration.test.js new file mode 100644 index 0000000..2c58f45 --- /dev/null +++ b/tests/integration/analyze-project-lib-prefix.integration.test.js @@ -0,0 +1,129 @@ +/** + * Regression test for the `lib/`-prefix correctness contract in + * `analyze_project` (2026-04-29). + * + * History — the previous version of this test pinned the inverse claim: + * that `'commands/X'` and `'lib/commands/X'` were both valid call forms. + * That assumption was wrong. platformOS resolves `function` paths under + * the partial search paths declared by `@platformos/platformos-common`: + * + * FILE_TYPE_DIRS[Partial] = ['views/partials', 'lib'] + * + * joined under `app/`. So `'commands/X'` is found at `app/lib/commands/X.liquid` + * and `'lib/commands/X'` is searched at `app/lib/lib/commands/X.liquid` — a + * directory that never exists in any sane project. Stripping the `lib/` + * prefix in `analyze-project.js` silently suppressed real errors AND + * matched the buggy stripping in `core/diagnostic-pipeline.js` / + * `error-enricher.js` / `core/rules/queries.js` / `fix-generator.js`, + * so the false assumption propagated end-to-end. + * + * The new contract: + * • `commands/X` (bare) is canonical; if the file exists, no issue. + * • `lib/commands/X` resolves to `app/lib/lib/commands/X.liquid`, + * which never exists, so analyze_project MUST flag it as a + * missing_command and surface the doubled `lib/lib/` path in the + * resolution string (so the agent sees what platformOS actually does). + * • A genuinely missing command (under the bare `commands/` form) is + * still reported with the canonical single-`lib/` resolution. + */ + +import { describe, it, expect, beforeAll, afterAll, setDefaultTimeout } from 'bun:test'; +import { writeFileSync, mkdirSync } from 'node:fs'; +import { join } from 'node:path'; +import { startServer, FIXTURE_DIR, createTempProject } from './helpers/server.js'; + +setDefaultTimeout(60_000); + +let server; +let proj; + +beforeAll(async () => { + proj = createTempProject(FIXTURE_DIR); + + // Real command + a build phase that the orchestrator calls under both + // shapes. The bare `commands/...` call must resolve cleanly; the + // `lib/commands/...` call must be flagged as wrong even though a file + // with the lib/-stripped name exists on disk. + const cmdDir = join(proj.dir, 'app/lib/commands/contacts/create'); + mkdirSync(cmdDir, { recursive: true }); + writeFileSync( + join(proj.dir, 'app/lib/commands/contacts/create.liquid'), + [ + '{% doc %}', + ' @param object {object} - input contact', + '{% enddoc %}', + '{% liquid', + " function object = 'commands/contacts/create/build', object: object", + " function object = 'lib/commands/contacts/create/build', object: object", + ' return object', + '%}', + '', + ].join('\n'), + 'utf8', + ); + writeFileSync( + join(cmdDir, 'build.liquid'), + "{% doc %}\n @param object {object}\n{% enddoc %}\n{% return object %}\n", + 'utf8', + ); + + // Page that calls a command which DOES NOT exist on disk — exercises + // the canonical missing-command path (no `lib/` prefix involved). + mkdirSync(join(proj.dir, 'app/views/pages/contacts'), { recursive: true }); + writeFileSync( + join(proj.dir, 'app/views/pages/contacts/test_miss.html.liquid'), + [ + '---', + 'slug: contacts/test-miss', + '---', + '{% liquid', + " function r = 'commands/contacts/never_written', object: context.params", + '%}', + '', + ].join('\n'), + 'utf8', + ); + + server = await startServer(proj.dir); +}); + +afterAll(() => { + server?.stop(); + proj?.cleanup(); +}); + +describe("analyze_project — `lib/` prefix is invalid, never optional", () => { + it('does NOT flag the bare `commands/X` form when the file exists at app/lib/commands/X.liquid', async () => { + const result = await server.callTool('analyze_project', {}); + const flagged = result.integrity.filter(i => + i.type === 'missing_command' && + (i.message ?? '').includes("'commands/contacts/create/build'") + ); + expect(flagged).toHaveLength(0); + }); + + it('FLAGS `lib/commands/X` as missing — the literal prefix expands to `app/lib/lib/...` and never resolves', async () => { + const result = await server.callTool('analyze_project', {}); + const flagged = result.integrity.filter(i => + i.type === 'missing_command' && + (i.message ?? '').includes("'lib/commands/contacts/create/build'") + ); + expect(flagged.length).toBeGreaterThan(0); + // The reported target path shows the doubled `lib/` so the agent sees + // exactly what platformOS would search at runtime. + expect(flagged[0].target).toBe('app/lib/lib/commands/contacts/create/build.liquid'); + expect(flagged[0].message).toContain('app/lib/lib/commands/contacts/create/build.liquid'); + }); + + it('still flags genuinely missing commands under the canonical (single-`lib/`) form', async () => { + const result = await server.callTool('analyze_project', {}); + const miss = result.integrity.filter(i => + i.type === 'missing_command' && + (i.message ?? '').includes('commands/contacts/never_written') + ); + expect(miss.length).toBeGreaterThan(0); + expect(miss[0].target).toBe('app/lib/commands/contacts/never_written.liquid'); + // No accidental doubling on the canonical form + expect(miss[0].target).not.toMatch(/app\/lib\/lib\//); + }); +}); diff --git a/tests/integration/cac/toggle.test.js b/tests/integration/cac/toggle.test.js new file mode 100644 index 0000000..63d8462 --- /dev/null +++ b/tests/integration/cac/toggle.test.js @@ -0,0 +1,178 @@ +/** + * CAC predictor — end-to-end toggle test. + * + * Verifies the opt-in 4th gating axis is wired correctly through: + * - server boot (defaults loaded, disabled by default) + * - HTTP endpoints (GET/POST /api/cac/config, GET /api/cac/decisions) + * - validate_code integration (no behavior change when disabled, + * decisions recorded in shadow mode, suppression in active mode) + * - hot-reload (POST changes take effect on the very next validate_code + * call without restart) + * + * The `force-disable-check.test.js` was the structural template for this + * test (POST → validate → assert → clear → re-assert). + */ + +import { describe, it, expect, beforeAll, afterAll, setDefaultTimeout } from 'bun:test'; +import { startServer, FIXTURE_DIR, createTempProject } from '../helpers/server.js'; +import { loadCacConfig } from '../../../src/core/cac-config.js'; + +setDefaultTimeout(60_000); + +let server; +let proj; + +// File chosen for the same reason force-disable-check.test.js picks it: pure +// HTML page with no partial renders → reliably triggers +// pos-supervisor:HtmlInPage. The CAC layer needs at least one diagnostic to +// chew on for shadow-mode telemetry to record an entry. +const FILE = 'app/views/pages/cac-toggle-test.liquid'; +const CONTENT = '---\nslug: cac-toggle-test\n---\n

hi

\n'; + +beforeAll(async () => { + proj = createTempProject(FIXTURE_DIR); + server = await startServer(proj.dir); +}); + +afterAll(() => { + server?.stop(); + proj?.cleanup(); +}); + +async function runValidate() { + return server.callTool('validate_code', { file_path: FILE, content: CONTENT, mode: 'quick' }); +} + +async function getCacConfig() { + const r = await fetch(server.baseUrl + '/api/cac/config'); + return r.json(); +} + +async function setCacConfig(patch) { + const r = await fetch(server.baseUrl + '/api/cac/config', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(patch), + }); + return { status: r.status, body: await r.json() }; +} + +async function getCacDecisions(limit = 50) { + const r = await fetch(server.baseUrl + `/api/cac/decisions?limit=${limit}`); + return r.json(); +} + +describe('CAC predictor: HTTP toggle + validate_code integration', () => { + it('GET /api/cac/config returns defaults; CAC is disabled out of the box', async () => { + const r = await getCacConfig(); + expect(r.config.enabled).toBe(false); + expect(r.config.mode).toBe('shadow'); + expect(r.defaults).toBeDefined(); + expect(Array.isArray(r.valid_modes)).toBe(true); + expect(r.valid_modes).toContain('shadow'); + expect(r.valid_modes).toContain('active'); + }); + + it('disabled: validate_code is unaffected; no decisions recorded', async () => { + // Sanity: predictor disabled means no entries appear from this call. + // Note: another describe block running first could have populated + // decisions, so we check the contract (count is finite + all entries + // come from earlier shadow/active runs only) by inspecting `summary`. + const before = await getCacDecisions(); + const beforeCount = before.count; + + const res = await runValidate(); + const all = [...(res.errors ?? []), ...(res.warnings ?? [])]; + expect(all.some(d => d.check === 'pos-supervisor:HtmlInPage')).toBe(true); + + const after = await getCacDecisions(); + // The disabled predictor must not append anything. + expect(after.count).toBe(beforeCount); + }); + + it('shadow mode: records decisions but does NOT modify diagnostics', async () => { + const setResp = await setCacConfig({ enabled: true, mode: 'shadow', threshold: 0.99, action: 'suppress' }); + expect(setResp.status).toBe(200); + expect(setResp.body.config.enabled).toBe(true); + expect(setResp.body.config.mode).toBe('shadow'); + // Persistence: the file should now exist on disk with the patched values. + const fromDisk = loadCacConfig(proj.dir); + expect(fromDisk.enabled).toBe(true); + expect(fromDisk.threshold).toBe(0.99); + + const before = await getCacDecisions(); + const beforeCount = before.count; + const res = await runValidate(); + + // Diagnostic still present — shadow mode never mutates result. + const all = [...(res.errors ?? []), ...(res.warnings ?? [])]; + expect(all.some(d => d.check === 'pos-supervisor:HtmlInPage')).toBe(true); + + // But decisions ring buffer grew. + const after = await getCacDecisions(); + expect(after.count).toBeGreaterThan(beforeCount); + // Each new decision is tagged with shadow mode. + const fresh = after.decisions.slice(beforeCount); + expect(fresh.every(d => d.mode === 'shadow')).toBe(true); + }); + + it('active mode + suppress: drops below-threshold diagnostics', async () => { + // Threshold 0.99 + action suppress: any diagnostic that has actual signal + // and predicts < 0.99 adoption gets dropped. With an empty analytics + // store the predictor falls to feature='prior' and ALWAYS allows. So we + // can't reliably suppress without seeded data — instead, assert the + // contract: when feature='prior', decision is 'allow' regardless of + // threshold. (Suppression on real data is covered by the unit tests in + // tests/unit/cac-predictor.test.js where we inject a deterministic + // historyProvider.) Here we verify only that flipping to active does not + // crash and does not over-suppress when there's no signal. + const setResp = await setCacConfig({ enabled: true, mode: 'active', threshold: 0.99, action: 'suppress' }); + expect(setResp.status).toBe(200); + expect(setResp.body.config.mode).toBe('active'); + + const res = await runValidate(); + const all = [...(res.errors ?? []), ...(res.warnings ?? [])]; + // Without analytics history every diagnostic falls to prior → allow. + expect(all.some(d => d.check === 'pos-supervisor:HtmlInPage')).toBe(true); + + const dec = await getCacDecisions(); + const recent = dec.decisions.at(-1); + expect(recent.mode).toBe('active'); + expect(['allow', 'suppress', 'downgrade']).toContain(recent.decision); + }); + + it('disabling resets behavior immediately (no restart needed)', async () => { + const setResp = await setCacConfig({ enabled: false }); + expect(setResp.status).toBe(200); + expect(setResp.body.config.enabled).toBe(false); + + const before = await getCacDecisions(); + const res = await runValidate(); + const all = [...(res.errors ?? []), ...(res.warnings ?? [])]; + expect(all.some(d => d.check === 'pos-supervisor:HtmlInPage')).toBe(true); + const after = await getCacDecisions(); + expect(after.count).toBe(before.count); // disabled: no append + }); + + it('POST with garbage body returns 400, leaves state untouched', async () => { + const r = await fetch(server.baseUrl + '/api/cac/config', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: 'not-json', + }); + expect([400, 500]).toContain(r.status); // body parser error → 400 (sometimes 500 from JSON) + }); + + it('POST with unknown keys: known fields applied, unknown silently dropped', async () => { + const setResp = await setCacConfig({ enabled: true, mode: 'shadow', threshold: 0.5, sneaky: 'no' }); + expect(setResp.status).toBe(200); + expect(setResp.body.config.threshold).toBe(0.5); + expect(setResp.body.config).not.toHaveProperty('sneaky'); + }); + + it('POST with out-of-range threshold gets coerced to default', async () => { + const setResp = await setCacConfig({ threshold: 99 }); + expect(setResp.status).toBe(200); + expect(setResp.body.config.threshold).toBeGreaterThan(0); + expect(setResp.body.config.threshold).toBeLessThan(1); + }); +}); diff --git a/tests/integration/event-replay.test.js b/tests/integration/event-replay.test.js new file mode 100644 index 0000000..0331e1e --- /dev/null +++ b/tests/integration/event-replay.test.js @@ -0,0 +1,135 @@ +/** + * Phase A1 acceptance gate — proves that replaying the on-disk events.ndjson + * through `applyEvent` reproduces the live projection the bus computed + * incrementally. + * + * Why this gate matters: the entire point of the event bus is to make the + * NDJSON log the canonical history. If replay-from-disk diverges from the + * write-through projection then the projection is lying — and every analytic + * built on top of it inherits the lie. This test runs the bus end-to-end + * against a real spawned server, exercises the projection-relevant kinds + * (server_start, pos_cli_resolved, lsp_event, index_event, tool_call), then + * shuts down cleanly and asserts the replayed state has the expected shape + * AND that the bus's own close-time invariant check did not log a mismatch. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'bun:test'; +import { join } from 'node:path'; +import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs'; +import { setTimeout as sleep } from 'node:timers/promises'; +import { startServer, FIXTURE_DIR, createTempProject } from './helpers/server.js'; +import { readEventLog } from '../../src/core/session-events.js'; +import { replay } from '../../src/core/session-state.js'; + +// Resolve the bus's session subdirectory. The legacy `saveSessionSummary` +// writes flat `session-*.json` files at the same level, so we filter for +// real directories. If multiple bus dirs survived from earlier runs, take +// the newest by mtime. +function findCurrentSessionDir(sessionsDir) { + const entries = readdirSync(sessionsDir) + .filter((n) => n.startsWith('session-')) + .map((n) => ({ name: n, full: join(sessionsDir, n) })) + .filter((e) => { + try { return statSync(e.full).isDirectory(); } catch { return false; } + }) + .map((e) => ({ ...e, mtime: statSync(e.full).mtimeMs })) + .sort((a, b) => b.mtime - a.mtime); + return entries[0]?.full ?? null; +} + +describe('event replay: acceptance gate', () => { + let project; + let server; + + beforeAll(async () => { + project = createTempProject(FIXTURE_DIR); + server = await startServer(project.dir); + }); + + afterAll(async () => { + server?.stop(); + // Give SIGTERM time to run shutdown() — sessionBus.close() must flush + // and run its final invariant check before we read the log. + await sleep(800); + project?.cleanup(); + }); + + it('records a startup sequence on disk that replays into a sane projection', async () => { + // Fire a few projection-relevant tool calls to populate by_tool + file_history. + await server.callTool('server_status', {}); + await server.callTool('project_map', { scope: 'lite' }); + + // Allow the tool_call events to flush through the bus. + await sleep(150); + + const sessionsDir = join(project.dir, '.pos-supervisor', 'sessions'); + expect(existsSync(sessionsDir)).toBe(true); + + const sessionDir = findCurrentSessionDir(sessionsDir); + expect(sessionDir).toBeTruthy(); + + const eventsPath = join(sessionDir, 'events.ndjson'); + expect(existsSync(eventsPath)).toBe(true); + + const events = readEventLog(eventsPath); + expect(events.length).toBeGreaterThan(0); + + // Envelope sanity — every record carries the required fields. + for (const e of events) { + expect(e.v).toBeGreaterThanOrEqual(1); + expect(typeof e.session_id).toBe('string'); + expect(typeof e.ts).toBe('string'); + expect(typeof e.kind).toBe('string'); + } + + // The session_id is consistent within a single boot. + const sids = new Set(events.map((e) => e.session_id)); + expect(sids.size).toBe(1); + + const replayed = replay(events); + + // server_start landed and projected. + expect(replayed.server.started_at).toBeTruthy(); + expect(replayed.server.version).toBeTruthy(); + expect(replayed.server.project_dir).toBe(project.dir); + + // tool_call rollups account for both calls we just made. + expect(replayed.by_tool.server_status?.calls ?? 0).toBeGreaterThanOrEqual(1); + expect(replayed.by_tool.project_map?.calls ?? 0).toBeGreaterThanOrEqual(1); + + // Event counter is monotonic with the log length (proves applyEvent + // increments _event_count for every record, including unknown kinds). + expect(replayed._event_count).toBe(events.length); + }); + + it('emits valid NDJSON — every line round-trips through readEvent', async () => { + const sessionsDir = join(project.dir, '.pos-supervisor', 'sessions'); + const sessionDir = findCurrentSessionDir(sessionsDir); + const eventsPath = join(sessionDir, 'events.ndjson'); + + const raw = readFileSync(eventsPath, 'utf-8').trim().split('\n'); + expect(raw.length).toBeGreaterThan(0); + + const errors = []; + readEventLog(eventsPath, { onError: (e) => errors.push(e) }); + expect(errors).toEqual([]); + }); + + it('shutdown writes server_stop with the SIGTERM reason', async () => { + server.stop(); + await sleep(800); + + const sessionsDir = join(project.dir, '.pos-supervisor', 'sessions'); + const sessionDir = findCurrentSessionDir(sessionsDir); + const eventsPath = join(sessionDir, 'events.ndjson'); + + const events = readEventLog(eventsPath); + const stop = events.find((e) => e.kind === 'server_stop'); + expect(stop).toBeDefined(); + expect(typeof stop.reason).toBe('string'); + + const replayed = replay(events); + expect(replayed.server.stopped).toBe(true); + expect(replayed.server.stop_reason).toBe(stop.reason); + }); +}); diff --git a/tests/integration/pos-cli/translation-array-index.test.js b/tests/integration/pos-cli/translation-array-index.test.js new file mode 100644 index 0000000..c2b3bd9 --- /dev/null +++ b/tests/integration/pos-cli/translation-array-index.test.js @@ -0,0 +1,102 @@ +/** + * Regression test for the fix-channel duplication bug (2026-04-25). + * + * Before this fix, an indexed translation key like `key[0]` produced THREE + * proposed_fixes per error: + * 1. A correct heuristic guidance ("Pass the full array, then iterate…"). + * 2. The rule engine's stale `suggest_nearest` guidance ("Replace with + * `en.parent.items`…") — Levenshtein found the parent key as the + * closest match, which is misleading. + * 3. Same as #2 for the second error. + * + * The fix: + * - The array-index case is now owned by a dedicated rule + * `TranslationKeyExists.array_index_misuse` (priority 5). + * - `suggest_nearest` and `create_key` are gated to NOT fire on indexed + * keys. + * - The validate-code merge loop drops heuristic GUIDANCE when a rule + * fix exists for the same diagnostic; heuristic TEXT_EDIT survives + * (actionable diff complements rule narrative). + * + * This test drives the full pipeline (validate_code, full mode) and + * asserts the agent sees ONLY the iteration guidance per error, not + * the misleading nearest-key suggestion. + */ + +import { it, expect, beforeAll, afterAll, setDefaultTimeout } from 'bun:test'; +import { describePosCli } from './guard.js'; +import { startServer, FIXTURE_DIR, createTempProject } from '../helpers/server.js'; + +setDefaultTimeout(30_000); + +let server; +let proj; + +beforeAll(async () => { + proj = createTempProject(FIXTURE_DIR); + server = await startServer(proj.dir); +}); + +afterAll(() => { + server?.stop(); + proj?.cleanup(); +}); + +// Using a key whose namespace exists in the fixture so the LSP actually +// fires `TranslationKeyExists`. `blog_posts.titl[0]` is close enough to the +// fixture's `blog_posts.title` that the parent key would have been the +// nearest-match candidate before the fix. Path is a flat partial — LSP +// translation resolution is path-sensitive in the fixture project, and +// nested subdirs sometimes skip the check. +const FILE = 'app/views/partials/test_arr.liquid'; +const CONTENT = "{{ 'blog_posts.titl[0]' | t }}"; + +describePosCli('translation array-index misuse: single-source-of-truth fix', () => { + it('emits ONE iteration guidance per error, never the misleading "nearest key" suggestion', async () => { + const result = await server.callTool('validate_code', { + file_path: FILE, + content: CONTENT, + mode: 'full', + }); + + const tkeDiags = [ + ...result.errors.filter(e => e.check === 'TranslationKeyExists'), + ...result.warnings.filter(w => w.check === 'TranslationKeyExists'), + ]; + expect(tkeDiags.length).toBeGreaterThanOrEqual(1); + + // Each diagnostic carries the array-index rule attribution. + for (const d of tkeDiags) { + expect(d.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + } + + // Filter proposed_fixes to ones from these TranslationKeyExists rows. + const tkeProposed = result.proposed_fixes.filter(f => + f.check === 'TranslationKeyExists' || /TranslationKeyExists/.test(f.rule_id ?? '') + ); + expect(tkeProposed.length).toBeGreaterThan(0); + + for (const f of tkeProposed) { + // Modern attribution wins for both rule and (any surviving) heuristic. + const id = f.rule_id ?? ''; + const isArrayRule = id === 'TranslationKeyExists.array_index_misuse'; + const isStrippedHeuristic = id.startsWith('heuristic:TranslationKeyExists'); + expect(isArrayRule || isStrippedHeuristic).toBe(true); + + // No fix should suggest the misleading "did you mean" or + // "Replace 'foo[0]' with 'foo'" rewrite. The bug we're guarding + // against was: the rule's stale `suggest_nearest` proposing the + // parent key as a "did you mean" candidate. + expect(f.description).not.toMatch(/[Dd]id you mean/); + expect(f.description).not.toMatch(/Replace `blog_posts\.titl\[\d+\]` with `blog_posts/); + } + + // Spot-check the iteration-guidance shape on at least one fix. + const arrayFix = tkeProposed.find(f => f.rule_id === 'TranslationKeyExists.array_index_misuse'); + expect(arrayFix).toBeDefined(); + expect(arrayFix.description).toMatch(/\{% for item in items %\}/); + expect(arrayFix.description).toMatch(/blog_posts\.titl/); + // arrayKey reference must NOT carry the [0]/[1] suffix. + expect(arrayFix.description).not.toMatch(/blog_posts\.titl\[\d+\]/); + }); +}); diff --git a/tests/integration/project-map.integration.test.js b/tests/integration/project-map.integration.test.js index 9d8f4d6..604e7d2 100644 --- a/tests/integration/project-map.integration.test.js +++ b/tests/integration/project-map.integration.test.js @@ -98,7 +98,7 @@ describe('project_map — full scope', () => { const result = await server.callTool('project_map', { scope: 'full' }); expect(result.summary.file_counts.schema).toBe(1); expect(result.summary.file_counts.graphql).toBe(4); - expect(result.summary.file_counts.pages).toBe(2); + expect(result.summary.file_counts.pages).toBe(3); expect(result.summary.file_counts.assets).toBeGreaterThan(0); }); diff --git a/tests/integration/validate-code-features.integration.test.js b/tests/integration/validate-code-features.integration.test.js index 4b3447f..32d2513 100644 --- a/tests/integration/validate-code-features.integration.test.js +++ b/tests/integration/validate-code-features.integration.test.js @@ -140,3 +140,43 @@ slug: test_gotchas_array } }); }); + +// --------------------------------------------------------------------------- +// Translation YAML structural validation +// --------------------------------------------------------------------------- + +describe('validate_code — translation YAML structural errors', () => { + it('reports TranslationMissingLocaleKey when top-level key is not a locale code', async () => { + const content = `enff:\n app:\n hello: "Hello"\n`; + const result = await server.callTool('validate_code', { + file_path: 'app/translations/en.yml', + content, + mode: 'quick', + }); + expect(result.errors.some(e => e.check === 'pos-supervisor:TranslationMissingLocaleKey')).toBe(true); + expect(result.status).toBe('error'); + expect(result.must_fix_before_write).toBe(true); + }); + + it('reports TranslationMissingLocaleKey when tree has no locale wrapper (app: at root)', async () => { + const content = `app:\n contact_form:\n title: "Contact"\n`; + const result = await server.callTool('validate_code', { + file_path: 'app/translations/en.yml', + content, + mode: 'quick', + }); + expect(result.errors.some(e => e.check === 'pos-supervisor:TranslationMissingLocaleKey')).toBe(true); + expect(result.status).toBe('error'); + }); + + it('passes a correctly wrapped translation file', async () => { + const content = `en:\n app:\n hello: "Hello"\n`; + const result = await server.callTool('validate_code', { + file_path: 'app/translations/en.yml', + content, + mode: 'quick', + }); + expect(result.errors.filter(e => e.check === 'pos-supervisor:TranslationMissingLocaleKey')).toHaveLength(0); + expect(result.status).not.toBe('error'); + }); +}); diff --git a/tests/unit/analytics-labels.test.js b/tests/unit/analytics-labels.test.js new file mode 100644 index 0000000..89de092 --- /dev/null +++ b/tests/unit/analytics-labels.test.js @@ -0,0 +1,234 @@ +import { describe, test, expect } from 'bun:test'; +import { + LABEL_MIN_OUTCOMES, + checkLabel, + ruleLabel, + harmfulSummary, + withCheckLabels, + withRuleLabels, +} from '../../src/core/analytics-labels.js'; + +// ── checkLabel ────────────────────────────────────────────────────────────── + +describe('checkLabel: sample-size gate', () => { + test('INSUFFICIENT_DATA when sample_size < threshold (the GraphQLVariablesCheck case)', () => { + // The exact pattern from the 2026-04-30 report: 4 outcomes, all regressed. + // Pre-gate this would have read "HARMFUL"; post-gate we refuse to label. + const card = { + check: 'GraphQLVariablesCheck', + sample_size: 4, + resolution_rate: { mean: 0, lower95: 0, upper95: 0 }, + mislead_rate: { mean: 1, lower95: 0.5, upper95: 1 }, + }; + expect(checkLabel(card)).toBe('INSUFFICIENT_DATA'); + }); + + test('INSUFFICIENT_DATA when sample_size === 0 (no outcomes at all)', () => { + expect(checkLabel({ sample_size: 0, resolution_rate: 0, mislead_rate: 0 })).toBe('INSUFFICIENT_DATA'); + }); + + test('INSUFFICIENT_DATA at threshold-minus-one', () => { + expect(checkLabel({ + sample_size: LABEL_MIN_OUTCOMES - 1, + resolution_rate: 1, mislead_rate: 0, + })).toBe('INSUFFICIENT_DATA'); + }); + + test('crosses gate at LABEL_MIN_OUTCOMES — same effectiveness now labelled', () => { + expect(checkLabel({ + sample_size: LABEL_MIN_OUTCOMES, + resolution_rate: 0.9, mislead_rate: 0.1, + })).toBe('GOOD'); + }); + + test('falls back to total_outcomes when sample_size missing', () => { + expect(checkLabel({ + total_outcomes: LABEL_MIN_OUTCOMES, + resolution_rate: 0.9, mislead_rate: 0.1, + })).toBe('GOOD'); + expect(checkLabel({ + total_outcomes: LABEL_MIN_OUTCOMES - 1, + resolution_rate: 0.9, mislead_rate: 0.1, + })).toBe('INSUFFICIENT_DATA'); + }); +}); + +describe('checkLabel: effectiveness buckets', () => { + // Once we're above the sample-size gate, the buckets must match the + // existing dashboard inline logic exactly so legacy reports don't shift. + const big = LABEL_MIN_OUTCOMES * 10; + + test('GOOD when effectiveness > 0.5', () => { + expect(checkLabel({ sample_size: big, resolution_rate: 0.9, mislead_rate: 0.1 })).toBe('GOOD'); + expect(checkLabel({ sample_size: big, resolution_rate: 0.7, mislead_rate: 0.0 })).toBe('GOOD'); + }); + + test('OK when 0.15 < effectiveness ≤ 0.5', () => { + expect(checkLabel({ sample_size: big, resolution_rate: 0.6, mislead_rate: 0.1 })).toBe('OK'); + expect(checkLabel({ sample_size: big, resolution_rate: 0.5, mislead_rate: 0.0 })).toBe('OK'); + }); + + test('LOW when 0 ≤ effectiveness ≤ 0.15', () => { + expect(checkLabel({ sample_size: big, resolution_rate: 0.2, mislead_rate: 0.1 })).toBe('LOW'); + expect(checkLabel({ sample_size: big, resolution_rate: 0.1, mislead_rate: 0.1 })).toBe('LOW'); + }); + + test('HARMFUL when effectiveness < 0', () => { + expect(checkLabel({ sample_size: big, resolution_rate: 0.1, mislead_rate: 0.5 })).toBe('HARMFUL'); + expect(checkLabel({ sample_size: big, resolution_rate: 0.0, mislead_rate: 0.6 })).toBe('HARMFUL'); + }); +}); + +describe('checkLabel: input shape tolerance', () => { + test('accepts Beta-posterior {mean,...} objects (server-side payload shape)', () => { + const card = { + sample_size: 20, + resolution_rate: { mean: 0.85, lower95: 0.7, upper95: 0.95 }, + mislead_rate: { mean: 0.05, lower95: 0.0, upper95: 0.15 }, + }; + expect(checkLabel(card)).toBe('GOOD'); + }); + + test('accepts bare numbers (legacy / test-shaped payloads)', () => { + expect(checkLabel({ + sample_size: 20, resolution_rate: 0.85, mislead_rate: 0.05, + })).toBe('GOOD'); + }); + + test('null/undefined card returns INSUFFICIENT_DATA without throwing', () => { + expect(checkLabel(null)).toBe('INSUFFICIENT_DATA'); + expect(checkLabel(undefined)).toBe('INSUFFICIENT_DATA'); + expect(checkLabel('not-an-object')).toBe('INSUFFICIENT_DATA'); + }); + + test('NaN sample_size is treated as zero (not crashed)', () => { + expect(checkLabel({ sample_size: NaN, resolution_rate: 1, mislead_rate: 0 })) + .toBe('INSUFFICIENT_DATA'); + }); +}); + +// ── ruleLabel ─────────────────────────────────────────────────────────────── + +describe('ruleLabel: precedence', () => { + test('UNMATCHED wins regardless of sample size', () => { + // Coverage gap is actionable on its own — one emit on a rule-less + // check still tells operators "write a rule for this". Sample-size + // gate must NOT mask it. + expect(ruleLabel({ unmatched: true, total_outcomes: 1, effectiveness: 0 })) + .toBe('UNMATCHED'); + expect(ruleLabel({ unmatched: true, total_outcomes: 100, effectiveness: 0.9 })) + .toBe('UNMATCHED'); + }); + + test('INSUFFICIENT_DATA when matched rule has < threshold outcomes', () => { + // The exact pattern from the 04-30 report: several AT-RISK rules with + // total_outcomes between 1 and 4. They become INSUFFICIENT_DATA. + expect(ruleLabel({ unmatched: false, total_outcomes: 4, effectiveness: -1 })) + .toBe('INSUFFICIENT_DATA'); + expect(ruleLabel({ unmatched: false, total_outcomes: 1, effectiveness: 0 })) + .toBe('INSUFFICIENT_DATA'); + }); + + test('AT RISK when effectiveness < 0.15 with enough samples', () => { + expect(ruleLabel({ unmatched: false, total_outcomes: 20, effectiveness: 0.1 })) + .toBe('AT RISK'); + expect(ruleLabel({ unmatched: false, total_outcomes: 20, effectiveness: -0.5 })) + .toBe('AT RISK'); + }); + + test('OK when effectiveness >= 0.15 with enough samples', () => { + expect(ruleLabel({ unmatched: false, total_outcomes: 20, effectiveness: 0.15 })) + .toBe('OK'); + expect(ruleLabel({ unmatched: false, total_outcomes: 20, effectiveness: 0.9 })) + .toBe('OK'); + }); + + test('null/undefined rule returns INSUFFICIENT_DATA without throwing', () => { + expect(ruleLabel(null)).toBe('INSUFFICIENT_DATA'); + expect(ruleLabel(undefined)).toBe('INSUFFICIENT_DATA'); + }); + + test('NaN effectiveness is INSUFFICIENT_DATA, not OK or AT RISK', () => { + expect(ruleLabel({ unmatched: false, total_outcomes: 20, effectiveness: NaN })) + .toBe('INSUFFICIENT_DATA'); + }); +}); + +// ── harmfulSummary ────────────────────────────────────────────────────────── + +describe('harmfulSummary', () => { + test('returns rows whose label is HARMFUL — never single-emit ghosts', () => { + const cards = [ + // Ghost: 4 emits all regressed. Pre-gate flagged HARMFUL; post-gate filtered. + { check: 'OldGhost', sample_size: 4, resolution_rate: 0, mislead_rate: 1 }, + // Real: 30 outcomes, true regression-heavy. + { check: 'RealHarm', sample_size: 30, resolution_rate: 0.1, mislead_rate: 0.6 }, + // Healthy: ignored. + { check: 'Healthy', sample_size: 30, resolution_rate: 0.9, mislead_rate: 0.05 }, + ]; + const harmful = harmfulSummary(cards); + expect(harmful).toHaveLength(1); + expect(harmful[0].check).toBe('RealHarm'); + }); + + test('non-array input returns []', () => { + expect(harmfulSummary(null)).toEqual([]); + expect(harmfulSummary(undefined)).toEqual([]); + expect(harmfulSummary('nope')).toEqual([]); + }); + + test('empty array returns []', () => { + expect(harmfulSummary([])).toEqual([]); + }); +}); + +// ── withCheckLabels / withRuleLabels ──────────────────────────────────────── + +describe('withCheckLabels', () => { + test('attaches .label without mutating input rows', () => { + const cards = [ + { check: 'A', sample_size: 30, resolution_rate: 0.9, mislead_rate: 0.05 }, + { check: 'B', sample_size: 2, resolution_rate: 0, mislead_rate: 1 }, + ]; + const out = withCheckLabels(cards); + expect(out).toHaveLength(2); + expect(out[0].label).toBe('GOOD'); + expect(out[1].label).toBe('INSUFFICIENT_DATA'); + // input untouched — no .label leakage + expect('label' in cards[0]).toBe(false); + expect('label' in cards[1]).toBe(false); + }); + + test('non-array input returns []', () => { + expect(withCheckLabels(null)).toEqual([]); + }); +}); + +describe('withRuleLabels', () => { + test('attaches .label without mutating input rows', () => { + const rules = [ + { rule_id: 'X.foo', unmatched: true, total_outcomes: 1, effectiveness: 0 }, + { rule_id: 'Y.bar', unmatched: false, total_outcomes: 30, effectiveness: 0.9 }, + { rule_id: 'Z.baz', unmatched: false, total_outcomes: 2, effectiveness: -1 }, + ]; + const out = withRuleLabels(rules); + expect(out[0].label).toBe('UNMATCHED'); + expect(out[1].label).toBe('OK'); + expect(out[2].label).toBe('INSUFFICIENT_DATA'); + expect('label' in rules[0]).toBe(false); + }); + + test('non-array input returns []', () => { + expect(withRuleLabels(undefined)).toEqual([]); + }); +}); + +// ── threshold export sanity ───────────────────────────────────────────────── + +describe('LABEL_MIN_OUTCOMES', () => { + test('exported as a positive integer ≥ 2', () => { + expect(typeof LABEL_MIN_OUTCOMES).toBe('number'); + expect(Number.isInteger(LABEL_MIN_OUTCOMES)).toBe(true); + expect(LABEL_MIN_OUTCOMES).toBeGreaterThanOrEqual(2); + }); +}); diff --git a/tests/unit/analytics-queries-k.test.js b/tests/unit/analytics-queries-k.test.js new file mode 100644 index 0000000..155dab7 --- /dev/null +++ b/tests/unit/analytics-queries-k.test.js @@ -0,0 +1,253 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { openAnalyticsStore } from '../../src/core/analytics-store.js'; +import { + diagnosticJourney, + confidenceCalibration, + fixAdoptionFunnel, + knowledgeGaps, +} from '../../src/core/analytics-queries.js'; +import { rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +function tmpPath() { + return join(tmpdir(), `pos-queries-k-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); +} + +function emitDiag(store, { fp, template_fp, session_id, check, confidence, file }) { + store.ingestEvent({ + v: 1, + session_id: session_id ?? 'sess-1', + ts: '2026-04-17T10:00:00Z', + kind: 'validator_emit', + fp, + template_fp, + file: file ?? 'test.liquid', + hint_rule_id: check ?? null, + confidence: confidence ?? null, + proposed_fixes: [], + }); +} + +let store; +let dbPath; + +beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); +}); + +afterEach(() => { + try { store.close(); } catch {} + try { rmSync(dbPath, { force: true }); } catch {} + try { rmSync(dbPath + '-wal', { force: true }); } catch {} + try { rmSync(dbPath + '-shm', { force: true }); } catch {} +}); + +// ── K2: Diagnostic Journey ────────────────────────────────────────────── + +describe('K2: diagnosticJourney', () => { + test('returns empty for unknown template_fp', () => { + const journey = diagnosticJourney(store, 'nonexistent'); + expect(journey.session_count).toBe(0); + expect(journey.timeline).toEqual([]); + expect(journey.check).toBeNull(); + }); + + test('returns journey across multiple sessions', () => { + emitDiag(store, { fp: 'fp1', template_fp: 'tpl-a', session_id: 's1', check: 'MissingPartial' }); + emitDiag(store, { fp: 'fp2', template_fp: 'tpl-a', session_id: 's2', check: 'MissingPartial' }); + emitDiag(store, { fp: 'fp3', template_fp: 'tpl-a', session_id: 's3', check: 'MissingPartial' }); + + const wid = store.insertWindow({ session_id: 's1', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z' }); + store.insertOutcome({ fp: 'fp1', window_id: wid, outcome: 'unchanged' }); + + const wid2 = store.insertWindow({ session_id: 's2', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T11:00:00Z', ts_end: '2026-04-17T11:01:00Z' }); + store.insertOutcome({ fp: 'fp2', window_id: wid2, outcome: 'resolved', fix_applied: 'verbatim' }); + + const journey = diagnosticJourney(store, 'tpl-a'); + expect(journey.check).toBe('MissingPartial'); + expect(journey.session_count).toBe(3); + expect(journey.timeline).toHaveLength(3); + + const s1 = journey.timeline.find(t => t.session_id === 's1'); + expect(s1.dominant_outcome).toBe('unchanged'); + expect(s1.rule_id).toBe('MissingPartial'); + + const s2 = journey.timeline.find(t => t.session_id === 's2'); + expect(s2.dominant_outcome).toBe('resolved'); + expect(s2.fix_applied).toBe('verbatim'); + + const s3 = journey.timeline.find(t => t.session_id === 's3'); + expect(s3.dominant_outcome).toBeNull(); + }); + + test('counts occurrences per session', () => { + emitDiag(store, { fp: 'fp1', template_fp: 'tpl-b', session_id: 's1', check: 'Check' }); + emitDiag(store, { fp: 'fp2', template_fp: 'tpl-b', session_id: 's1', check: 'Check' }); + emitDiag(store, { fp: 'fp3', template_fp: 'tpl-b', session_id: 's1', check: 'Check' }); + + const journey = diagnosticJourney(store, 'tpl-b'); + expect(journey.timeline).toHaveLength(1); + expect(journey.timeline[0].occurrences).toBe(3); + }); +}); + +// ── K3: Confidence Calibration ────────────────────────────────────────── + +describe('K3: confidenceCalibration', () => { + test('returns empty when no diagnostics have confidence', () => { + emitDiag(store, { fp: 'fp1', template_fp: 'tpl', session_id: 's1', check: 'X' }); + const wid = store.insertWindow({ session_id: 's1', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z' }); + store.insertOutcome({ fp: 'fp1', window_id: wid, outcome: 'resolved' }); + + const cal = confidenceCalibration(store); + expect(cal).toEqual([]); + }); + + test('buckets diagnostics by confidence and computes resolution rate', () => { + const wid = store.insertWindow({ session_id: 's1', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z' }); + + for (let i = 0; i < 10; i++) { + emitDiag(store, { fp: `hi-${i}`, template_fp: 'tpl', session_id: 's1', check: 'X', confidence: 0.85 }); + store.insertOutcome({ fp: `hi-${i}`, window_id: wid, outcome: i < 8 ? 'resolved' : 'unchanged' }); + } + for (let i = 0; i < 10; i++) { + emitDiag(store, { fp: `lo-${i}`, template_fp: 'tpl', session_id: 's1', check: 'X', confidence: 0.25 }); + store.insertOutcome({ fp: `lo-${i}`, window_id: wid, outcome: i < 3 ? 'resolved' : 'unchanged' }); + } + + const cal = confidenceCalibration(store, { buckets: 10 }); + expect(cal.length).toBeGreaterThanOrEqual(2); + + const highBucket = cal.find(b => b.predicted >= 0.8); + const lowBucket = cal.find(b => b.predicted <= 0.3); + expect(highBucket).toBeDefined(); + expect(lowBucket).toBeDefined(); + expect(highBucket.actual_resolution).toBeGreaterThan(lowBucket.actual_resolution); + }); + + test('respects custom bucket count', () => { + const wid = store.insertWindow({ session_id: 's1', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z' }); + + for (let i = 0; i < 20; i++) { + const conf = (i % 5) * 0.2 + 0.1; + emitDiag(store, { fp: `fp-${i}`, template_fp: 'tpl', session_id: 's1', check: 'X', confidence: conf }); + store.insertOutcome({ fp: `fp-${i}`, window_id: wid, outcome: 'resolved' }); + } + + const cal5 = confidenceCalibration(store, { buckets: 5 }); + expect(cal5.length).toBeLessThanOrEqual(5); + }); + + test('includes pipeline-stamped default confidences alongside rule-scored ones (A2)', () => { + const wid = store.insertWindow({ session_id: 's1', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z' }); + + // Rule-scored high-confidence row — resolved. + emitDiag(store, { fp: 'rule-1', template_fp: 'tpl', session_id: 's1', check: 'X', confidence: 0.92 }); + store.insertOutcome({ fp: 'rule-1', window_id: wid, outcome: 'resolved' }); + + // Pipeline-default warning (0.7) — unchanged. Pre-A2 this row would have + // been excluded by the NULL filter in the query; post-A2 every surviving + // diagnostic carries a default, so the bucket sees it. + emitDiag(store, { fp: 'default-1', template_fp: 'tpl', session_id: 's1', check: 'X', confidence: 0.7 }); + store.insertOutcome({ fp: 'default-1', window_id: wid, outcome: 'unchanged' }); + + const cal = confidenceCalibration(store, { buckets: 10 }); + const totalSample = cal.reduce((sum, b) => sum + b.sample_size, 0); + expect(totalSample).toBe(2); + + const mid = cal.find(b => b.predicted >= 0.6 && b.predicted <= 0.8); + expect(mid?.sample_size).toBe(1); + expect(mid?.actual_resolution).toBe(0); + }); +}); + +// ── K4: Fix Adoption Funnel ───────────────────────────────────────────── + +describe('K4: fixAdoptionFunnel', () => { + test('returns zeroes for empty store', () => { + const funnel = fixAdoptionFunnel(store); + expect(funnel.emitted).toBe(0); + expect(funnel.rule_matched).toBe(0); + expect(funnel.fix_proposed).toBe(0); + expect(funnel.resolved).toBe(0); + }); + + test('computes full funnel with data', () => { + for (let i = 0; i < 20; i++) { + const check = i < 15 ? 'TestCheck' : null; + emitDiag(store, { fp: `fp-${i}`, template_fp: 'tpl', session_id: 's1', check }); + } + + for (let i = 0; i < 10; i++) { + store.db.prepare(` + INSERT INTO proposed_fixes (fp, session_id, ts, kind) VALUES (?, ?, ?, ?) + `).run(`fp-${i}`, 's1', '2026-04-17T10:00:00Z', 'replace'); + } + + const wid = store.insertWindow({ session_id: 's1', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z' }); + + for (let i = 0; i < 8; i++) { + store.insertOutcome({ + fp: `fp-${i}`, window_id: wid, + outcome: i < 5 ? 'resolved' : 'unchanged', + fix_applied: i < 3 ? 'verbatim' : (i < 5 ? 'partial' : null), + }); + } + store.insertOutcome({ fp: 'fp-8', window_id: wid, outcome: 'regressed' }); + + const funnel = fixAdoptionFunnel(store); + expect(funnel.emitted).toBe(20); + expect(funnel.rule_matched).toBe(15); + expect(funnel.fix_proposed).toBe(10); + expect(funnel.fix_adopted_verbatim).toBe(3); + expect(funnel.fix_adopted_partial).toBe(2); + expect(funnel.resolved).toBe(5); + expect(funnel.regressed).toBe(1); + }); +}); + +// ── K5: Knowledge Gaps ────────────────────────────────────────────────── + +describe('K5: knowledgeGaps', () => { + test('returns empty for insufficient data', () => { + emitDiag(store, { fp: 'fp1', template_fp: 'tpl', session_id: 's1', check: 'X' }); + const gaps = knowledgeGaps(store); + expect(gaps).toEqual([]); + }); + + test('identifies checks with low rule coverage', () => { + for (let i = 0; i < 10; i++) { + emitDiag(store, { fp: `covered-${i}`, template_fp: 'tpl-c', session_id: 's1', check: 'CoveredCheck' }); + } + for (let i = 0; i < 10; i++) { + emitDiag(store, { fp: `uncov-${i}`, template_fp: 'tpl-u', session_id: 's1', check: i < 3 ? 'UncoveredCheck' : null }); + } + + const gaps = knowledgeGaps(store); + const covered = gaps.find(g => g.check === 'CoveredCheck'); + const uncovered = gaps.find(g => g.check === 'unknown'); + + expect(covered).toBeDefined(); + expect(covered.coverage_rate).toBe(1); + expect(uncovered).toBeDefined(); + expect(uncovered.unmatched_count).toBe(7); + }); + + test('includes resolution rate per check', () => { + for (let i = 0; i < 5; i++) { + emitDiag(store, { fp: `fp-${i}`, template_fp: 'tpl', session_id: 's1', check: 'ResCheck' }); + } + + const wid = store.insertWindow({ session_id: 's1', file: 'test.liquid', idx: 0, ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z' }); + for (let i = 0; i < 5; i++) { + store.insertOutcome({ fp: `fp-${i}`, window_id: wid, outcome: i < 4 ? 'resolved' : 'unchanged' }); + } + + const gaps = knowledgeGaps(store); + const resCheck = gaps.find(g => g.check === 'ResCheck'); + expect(resCheck).toBeDefined(); + expect(resCheck.avg_resolution_rate).toBe(0.8); + }); +}); diff --git a/tests/unit/analytics-queries.test.js b/tests/unit/analytics-queries.test.js new file mode 100644 index 0000000..bb19e66 --- /dev/null +++ b/tests/unit/analytics-queries.test.js @@ -0,0 +1,813 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { openAnalyticsStore } from '../../src/core/analytics-store.js'; +import { + betaPosterior, + checkScorecards, + toolSequenceBigrams, + sessionSummaries, + recommendations, + rulePerformance, + diagnosticJourney, + ruleDrilldown, + fixAdoptionFunnel, + adaptiveModeImpact, + fixRulePerformance, + ruleScoresByCategory, + knowledgeGaps, + confidenceCalibration, +} from '../../src/core/analytics-queries.js'; +import { rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +function tmpPath() { + return join(tmpdir(), `pos-queries-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); +} + +function emitEvent(store, sessionId, fp, check, ts = '2026-04-17T10:00:00Z') { + store.ingestEvent({ + v: 1, session_id: sessionId, ts, kind: 'validator_emit', + fp, file: 'app/views/pages/index.html.liquid', + hint_rule_id: check, proposed_fixes: [], + }); +} + +function toolCallEvent(store, sessionId, tool, ts = '2026-04-17T10:00:00Z') { + store.ingestEvent({ + v: 1, session_id: sessionId, ts, kind: 'tool_call', + tool, duration_ms: 100, success: true, + }); +} + +let store; +let dbPath; + +beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); +}); + +afterEach(() => { + try { store.close(); } catch {} + try { rmSync(dbPath, { force: true }); } catch {} + try { rmSync(dbPath + '-wal', { force: true }); } catch {} + try { rmSync(dbPath + '-shm', { force: true }); } catch {} +}); + +describe('betaPosterior', () => { + test('uniform prior with no data returns ~0.5', () => { + const { mean } = betaPosterior(0, 0); + expect(mean).toBeCloseTo(0.5, 2); + }); + + test('all successes → high mean', () => { + const { mean, lower95 } = betaPosterior(20, 20); + expect(mean).toBeGreaterThan(0.85); + expect(lower95).toBeGreaterThan(0.7); + }); + + test('no successes → low mean', () => { + const { mean, upper95 } = betaPosterior(0, 20); + expect(mean).toBeLessThan(0.15); + expect(upper95).toBeLessThan(0.3); + }); + + test('50/50 data → mean near 0.5', () => { + const { mean } = betaPosterior(50, 100); + expect(mean).toBeCloseTo(0.5, 1); + }); + + test('confidence interval narrows with more data', () => { + const small = betaPosterior(5, 10); + const large = betaPosterior(50, 100); + const smallWidth = small.upper95 - small.lower95; + const largeWidth = large.upper95 - large.lower95; + expect(largeWidth).toBeLessThan(smallWidth); + }); +}); + +describe('checkScorecards', () => { + test('returns empty for insufficient data', () => { + for (let i = 0; i < 5; i++) { + emitEvent(store, 's1', `fp${i}`, 'MissingPartial', `2026-04-17T10:0${i}:00Z`); + } + const cards = checkScorecards(store, { minCohort: 10 }); + expect(cards).toHaveLength(0); + }); + + test('returns scorecard when cohort met', () => { + for (let i = 0; i < 12; i++) { + emitEvent(store, 's1', `fp${i}`, 'MissingPartial', `2026-04-17T10:${String(i).padStart(2, '0')}:00Z`); + } + const cards = checkScorecards(store, { minCohort: 10 }); + expect(cards).toHaveLength(1); + expect(cards[0].check).toBe('MissingPartial'); + expect(cards[0].emitted).toBe(12); + }); + + test('includes outcome rates when outcomes exist', () => { + for (let i = 0; i < 15; i++) { + emitEvent(store, 's1', `fp${i}`, 'UndefinedObject', `2026-04-17T10:${String(i).padStart(2, '0')}:00Z`); + } + + const windowId = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:15:00Z', + }); + + for (let i = 0; i < 10; i++) { + store.insertOutcome({ + fp: `fp${i}`, window_id: windowId, + outcome: i < 7 ? 'resolved' : 'unchanged', + }); + } + + const cards = checkScorecards(store, { minCohort: 10 }); + expect(cards).toHaveLength(1); + expect(cards[0].resolution_rate.mean).toBeGreaterThan(0.5); + expect(cards[0].sample_size).toBe(10); + }); + + test('filters by sessionId', () => { + for (let i = 0; i < 12; i++) { + emitEvent(store, 's1', `a${i}`, 'CheckA', `2026-04-17T10:${String(i).padStart(2, '0')}:00Z`); + } + for (let i = 0; i < 12; i++) { + emitEvent(store, 's2', `b${i}`, 'CheckA', `2026-04-17T11:${String(i).padStart(2, '0')}:00Z`); + } + + const allCards = checkScorecards(store, { minCohort: 10 }); + expect(allCards).toHaveLength(1); + expect(allCards[0].emitted).toBe(24); + + const s1Cards = checkScorecards(store, { minCohort: 10, sessionId: 's1' }); + expect(s1Cards).toHaveLength(1); + expect(s1Cards[0].emitted).toBe(12); + }); +}); + +describe('toolSequenceBigrams', () => { + test('computes bigrams from tool calls', () => { + toolCallEvent(store, 's1', 'project_map', '2026-04-17T10:00:00Z'); + toolCallEvent(store, 's1', 'scaffold', '2026-04-17T10:01:00Z'); + toolCallEvent(store, 's1', 'validate_intent', '2026-04-17T10:02:00Z'); + toolCallEvent(store, 's1', 'validate_code', '2026-04-17T10:03:00Z'); + toolCallEvent(store, 's1', 'validate_code', '2026-04-17T10:04:00Z'); + + const bigrams = toolSequenceBigrams(store, { sessionId: 's1' }); + expect(bigrams.length).toBeGreaterThan(0); + + const pmScaffold = bigrams.find(b => b.bigram[0] === 'project_map' && b.bigram[1] === 'scaffold'); + expect(pmScaffold).toBeDefined(); + expect(pmScaffold.count).toBe(1); + expect(pmScaffold.confidence).toBeGreaterThan(0); + }); + + test('returns empty for < 2 calls', () => { + toolCallEvent(store, 's1', 'validate_code', '2026-04-17T10:00:00Z'); + expect(toolSequenceBigrams(store, { sessionId: 's1' })).toHaveLength(0); + }); + + test('repeated sequences have higher counts', () => { + for (let i = 0; i < 5; i++) { + toolCallEvent(store, 's1', 'validate_code', `2026-04-17T10:${String(i * 2).padStart(2, '0')}:00Z`); + toolCallEvent(store, 's1', 'validate_code', `2026-04-17T10:${String(i * 2 + 1).padStart(2, '0')}:00Z`); + } + const bigrams = toolSequenceBigrams(store, { sessionId: 's1' }); + const vcVc = bigrams.find(b => b.bigram[0] === 'validate_code' && b.bigram[1] === 'validate_code'); + expect(vcVc).toBeDefined(); + expect(vcVc.count).toBeGreaterThanOrEqual(5); + }); +}); + +describe('sessionSummaries', () => { + test('returns summary per session', () => { + store.ingestEvent({ + v: 1, session_id: 's1', ts: '2026-04-17T10:00:00Z', + kind: 'server_start', project_dir: '/tmp', version: '1.0', started_at: '2026-04-17T10:00:00Z', + }); + toolCallEvent(store, 's1', 'validate_code', '2026-04-17T10:01:00Z'); + toolCallEvent(store, 's1', 'validate_intent', '2026-04-17T10:02:00Z'); + emitEvent(store, 's1', 'fp1', 'Check1', '2026-04-17T10:03:00Z'); + + const summaries = sessionSummaries(store); + expect(summaries).toHaveLength(1); + expect(summaries[0].session_id).toBe('s1'); + expect(summaries[0].event_count).toBe(4); + expect(summaries[0].tool_calls).toBe(2); + expect(summaries[0].validate_code_calls).toBe(1); + expect(summaries[0].used_validate_intent).toBe(true); + expect(summaries[0].diagnostics_emitted).toBe(1); + }); + + test('detects when validate_intent is not used', () => { + toolCallEvent(store, 's2', 'validate_code', '2026-04-17T10:00:00Z'); + const summaries = sessionSummaries(store); + const s = summaries.find(s => s.session_id === 's2'); + expect(s.used_validate_intent).toBe(false); + }); +}); + +describe('recommendations', () => { + test('flags checks with high mislead rate', () => { + for (let i = 0; i < 20; i++) { + emitEvent(store, 's1', `fp${i}`, 'BadCheck', `2026-04-17T10:${String(i).padStart(2, '0')}:00Z`); + } + + const windowId = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:20:00Z', + }); + + for (let i = 0; i < 15; i++) { + store.insertOutcome({ + fp: `fp${i}`, window_id: windowId, + outcome: i < 10 ? 'regressed' : 'resolved', + }); + } + + const recs = recommendations(store, 0.3); + expect(recs.length).toBeGreaterThan(0); + expect(recs[0].check).toBe('BadCheck'); + expect(recs[0].recommendation).toContain('misleads'); + }); + + test('returns empty when no checks exceed threshold', () => { + for (let i = 0; i < 20; i++) { + emitEvent(store, 's1', `fp${i}`, 'GoodCheck', `2026-04-17T10:${String(i).padStart(2, '0')}:00Z`); + } + + const windowId = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:20:00Z', + }); + + for (let i = 0; i < 15; i++) { + store.insertOutcome({ + fp: `fp${i}`, window_id: windowId, + outcome: i < 12 ? 'resolved' : 'unchanged', + }); + } + + const recs = recommendations(store, 0.3); + expect(recs).toHaveLength(0); + }); +}); + +// ── A4: rulePerformance (reporting view) ─────────────────────────────────── + +describe('rulePerformance', () => { + function emitWithRule(store, { fp, sessionId, check, ruleId, file, ts }) { + store.ingestEvent({ + v: 1, + session_id: sessionId, + ts: ts ?? '2026-04-17T10:00:00Z', + kind: 'validator_emit', + fp, + template_fp: `tpl-${fp}`, + file: file ?? 'app/views/pages/index.html.liquid', + check, + hint_rule_id: ruleId, + proposed_fixes: [], + }); + } + + test('returns empty when no rule_ids present', () => { + expect(rulePerformance(store)).toEqual([]); + }); + + test('default threshold of 1 surfaces single-emit rules', () => { + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'UnknownFilter', ruleId: 'UnknownFilter.typo' }); + const out = rulePerformance(store); + expect(out).toHaveLength(1); + expect(out[0].rule_id).toBe('UnknownFilter.typo'); + expect(out[0].emitted).toBe(1); + expect(out[0].unmatched).toBe(false); + }); + + test('includes `${check}.unmatched` fallback rule_ids (reporting coverage)', () => { + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'MissingPartial', ruleId: 'MissingPartial.unmatched' }); + emitWithRule(store, { fp: 'fp2', sessionId: 's1', check: 'MissingPartial', ruleId: 'MissingPartial.unmatched' }); + const out = rulePerformance(store); + const row = out.find(r => r.rule_id === 'MissingPartial.unmatched'); + expect(row).toBeDefined(); + expect(row.unmatched).toBe(true); + expect(row.emitted).toBe(2); + }); + + test('excludes rule_id "unknown" sentinel', () => { + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'X', ruleId: 'unknown' }); + expect(rulePerformance(store)).toEqual([]); + }); + + test('minEmitted filter honoured', () => { + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'A', ruleId: 'A.x' }); + emitWithRule(store, { fp: 'fp2', sessionId: 's1', check: 'B', ruleId: 'B.y' }); + emitWithRule(store, { fp: 'fp3', sessionId: 's1', check: 'B', ruleId: 'B.y' }); + const out = rulePerformance(store, { minEmitted: 2 }); + expect(out.map(r => r.rule_id)).toEqual(['B.y']); + }); + + test('EXISTS join does not inflate outcomes by per-emit diagnostic rows', () => { + // Post-A1: outcomes row per (session, file, fp). The same fp may appear + // many times in diagnostics (each validator_emit event). rulePerformance + // must count each outcome row once, not per emitting diagnostic. + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'UnknownFilter', ruleId: 'UnknownFilter.typo', ts: '2026-04-17T10:00:00Z' }); + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'UnknownFilter', ruleId: 'UnknownFilter.typo', ts: '2026-04-17T10:01:00Z' }); + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'UnknownFilter', ruleId: 'UnknownFilter.typo', ts: '2026-04-17T10:02:00Z' }); + + const windowId = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:03:00Z', + }); + store.insertOutcome({ fp: 'fp1', window_id: windowId, outcome: 'resolved', fix_applied: 'verbatim' }); + + const out = rulePerformance(store); + const row = out.find(r => r.rule_id === 'UnknownFilter.typo'); + expect(row.total_outcomes).toBe(1); + expect(row.resolved).toBe(1); + expect(row.adopted).toBe(1); + }); + + test('does not expose `disabled` flag (reporting is not a promotion decision)', () => { + emitWithRule(store, { fp: 'fp1', sessionId: 's1', check: 'A', ruleId: 'A.x' }); + const out = rulePerformance(store); + expect(out[0].disabled).toBeUndefined(); + }); +}); + +// ── hint_md_hash surfaced by journey + drilldown queries (Bug 2) ──────────── + +describe('journey + drilldown surface hint_md_hash', () => { + test('diagnosticJourney timeline entries carry hint_md_hash', () => { + store.ingestEvent({ + v: 1, session_id: 'j1', ts: '2026-04-17T10:00:00Z', kind: 'validator_emit', + fp: 'fp-j1', template_fp: 'tpl-j1', file: 'app/views/pages/a.liquid', + hint_rule_id: 'Check.rule', hint_md_hash: 'x'.repeat(64), + content_hash: 'y'.repeat(64), proposed_fixes: [], + }); + + const j = diagnosticJourney(store, 'tpl-j1'); + expect(j.timeline).toHaveLength(1); + expect(j.timeline[0].hint_md_hash).toBe('x'.repeat(64)); + }); + + test('ruleDrilldown samples carry hint_md_hash', () => { + store.ingestEvent({ + v: 1, session_id: 'd1', ts: '2026-04-17T10:00:00Z', kind: 'validator_emit', + fp: 'fp-d1', template_fp: 'tpl-d1', file: 'app/views/pages/b.liquid', + hint_rule_id: 'Check.rule', hint_md_hash: 'a'.repeat(64), + content_hash: 'b'.repeat(64), proposed_fixes: [], + }); + + const d = ruleDrilldown(store, 'Check.rule'); + expect(d.samples).toHaveLength(1); + expect(d.samples[0].hint_md_hash).toBe('a'.repeat(64)); + }); +}); + +// ── Funnel adoption counts (Bug 1) ────────────────────────────────────────── + +describe('fixAdoptionFunnel counts fix_applied correctly', () => { + test('picks up verbatim + partial + null adoption buckets', () => { + // Seed three fps, each with one diagnostic and one outcome that carries a + // fix_applied label. Post-Bug-1: classifyAndStoreWindows writes these; + // the query is read-only. + store.ingestEvent({ + v: 1, session_id: 'fa', ts: '2026-04-17T10:00:00Z', kind: 'validator_emit', + fp: 'v', template_fp: 'tpl-v', file: 'app/views/pages/v.liquid', + hint_rule_id: 'C.r', proposed_fixes: [], + }); + store.ingestEvent({ + v: 1, session_id: 'fa', ts: '2026-04-17T10:00:00Z', kind: 'validator_emit', + fp: 'p', template_fp: 'tpl-p', file: 'app/views/pages/p.liquid', + hint_rule_id: 'C.r', proposed_fixes: [], + }); + store.ingestEvent({ + v: 1, session_id: 'fa', ts: '2026-04-17T10:00:00Z', kind: 'validator_emit', + fp: 'n', template_fp: 'tpl-n', file: 'app/views/pages/n.liquid', + hint_rule_id: 'C.r', proposed_fixes: [], + }); + + const w1 = store.insertWindow({ session_id: 'fa', file: 'app/views/pages/v.liquid', idx: 0, ts_start: 'a', ts_end: 'b' }); + const w2 = store.insertWindow({ session_id: 'fa', file: 'app/views/pages/p.liquid', idx: 0, ts_start: 'a', ts_end: 'b' }); + const w3 = store.insertWindow({ session_id: 'fa', file: 'app/views/pages/n.liquid', idx: 0, ts_start: 'a', ts_end: 'b' }); + store.insertOutcome({ fp: 'v', window_id: w1, outcome: 'resolved', fix_applied: 'verbatim' }); + store.insertOutcome({ fp: 'p', window_id: w2, outcome: 'resolved', fix_applied: 'partial' }); + store.insertOutcome({ fp: 'n', window_id: w3, outcome: 'resolved', fix_applied: null }); + + const f = fixAdoptionFunnel(store); + expect(f.fix_adopted_verbatim).toBe(1); + expect(f.fix_adopted_partial).toBe(1); + expect(f.resolved).toBe(3); + }); +}); + +// ── Part G — adaptiveModeImpact window query ──────────────────────────────── + +describe('adaptiveModeImpact', () => { + const NOW = Date.now(); + function emitAt(msAgo, overrides = {}) { + const ts = new Date(NOW - msAgo).toISOString(); + store.ingestEvent({ + v: 1, session_id: 'ami', ts, kind: 'validator_emit', + fp: overrides.fp ?? 'fp-' + msAgo, + template_fp: 'tpl', file: 'app/views/pages/x.liquid', + hint_rule_id: overrides.rule ?? 'X.r', + confidence: overrides.confidence ?? 0.8, + proposed_fixes: [], + }); + } + + test('returns zero counts when window is empty', () => { + const r = adaptiveModeImpact(store, { windowMs: 60_000 }); + expect(r.emits_in_window).toBe(0); + expect(r.rule_matched_in_window).toBe(0); + expect(r.confidence.samples).toBe(0); + }); + + test('counts emits within the window + excludes .unmatched from rule_matched', () => { + emitAt(1_000, { fp: 'a', rule: 'X.r' }); + emitAt(2_000, { fp: 'b', rule: 'X.r' }); + emitAt(3_000, { fp: 'c', rule: 'Y.unmatched' }); + emitAt(3_600_001, { fp: 'old', rule: 'X.r' }); // outside 1h window + + const r = adaptiveModeImpact(store, { windowMs: 3_600_000 }); + expect(r.emits_in_window).toBe(3); + expect(r.rule_matched_in_window).toBe(2); // 'Y.unmatched' excluded + }); + + test('emits_by_rule groups per rule_id; caller intersects with disabled set', () => { + emitAt(1_000, { fp: 'a', rule: 'DisA' }); + emitAt(2_000, { fp: 'b', rule: 'DisA' }); + emitAt(2_000, { fp: 'c', rule: 'DisB' }); + + const r = adaptiveModeImpact(store, { windowMs: 60_000 }); + expect(r.emits_by_rule).toEqual({ DisA: 2, DisB: 1 }); + }); + + test('confidence stats reflect window-scoped emits only', () => { + emitAt(1_000, { fp: 'a', confidence: 0.9 }); + emitAt(2_000, { fp: 'b', confidence: 0.5 }); + + const r = adaptiveModeImpact(store, { windowMs: 60_000 }); + expect(r.confidence.samples).toBe(2); + expect(r.confidence.mean).toBeCloseTo(0.7, 2); + expect(r.confidence.min).toBe(0.5); + expect(r.confidence.max).toBe(0.9); + }); +}); + +// ── I1 — fixRulePerformance (attribution by proposed_fixes.rule_id) ───────── + +describe('fixRulePerformance', () => { + function emitWithFixes(sessionId, fp, fixes) { + store.ingestEvent({ + v: 1, session_id: sessionId, ts: '2026-04-17T10:00:00Z', kind: 'validator_emit', + fp, template_fp: 'tpl', file: 'app/views/pages/x.liquid', + hint_rule_id: 'Ignored', proposed_fixes: fixes, + }); + } + function outcome(fp, sessionId, out, fixApplied = null) { + const wid = store.insertWindow({ + session_id: sessionId, file: 'app/views/pages/x.liquid', idx: 0, + ts_start: 'a', ts_end: 'b', + }); + store.insertOutcome({ fp, window_id: wid, outcome: out, fix_applied: fixApplied }); + } + + test('returns empty when no rule_ids on fixes', () => { + emitWithFixes('s1', 'f', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: null }]); + expect(fixRulePerformance(store)).toEqual([]); + }); + + test('groups rule-engine vs heuristic under a `source` field', () => { + emitWithFixes('s1', 'f1', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'UnknownFilter.suggest_nearest' }]); + emitWithFixes('s1', 'f2', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'heuristic:UnknownFilter.text_edit' }]); + const out = fixRulePerformance(store); + const rule = out.find(r => r.rule_id === 'UnknownFilter.suggest_nearest'); + const heur = out.find(r => r.rule_id === 'heuristic:UnknownFilter.text_edit'); + expect(rule?.source).toBe('rule'); + expect(heur?.source).toBe('heuristic'); + }); + + test('aggregates adoption + resolution per rule_id', () => { + emitWithFixes('s1', 'a', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'heuristic:UnknownFilter.text_edit' }]); + emitWithFixes('s2', 'b', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'heuristic:UnknownFilter.text_edit' }]); + emitWithFixes('s3', 'c', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'heuristic:UnknownFilter.text_edit' }]); + outcome('a', 's1', 'resolved', 'verbatim'); + outcome('b', 's2', 'resolved', 'partial'); + outcome('c', 's3', 'unchanged', null); + + const r = fixRulePerformance(store).find(r => r.rule_id === 'heuristic:UnknownFilter.text_edit'); + expect(r.outcomes).toBe(3); + expect(r.adopted_verbatim).toBe(1); + expect(r.adopted_partial).toBe(1); + expect(r.adoption_rate).toBeCloseTo(2 / 3, 2); + expect(r.resolution_rate).toBeCloseTo(2 / 3, 2); + }); + + test('minProposed filter honoured', () => { + emitWithFixes('s1', 'a', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'heuristic:Rare.text_edit' }]); + emitWithFixes('s2', 'b', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'heuristic:Common.text_edit' }]); + emitWithFixes('s3', 'c', [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'heuristic:Common.text_edit' }]); + const out = fixRulePerformance(store, { minProposed: 2 }); + expect(out.map(r => r.rule_id)).toEqual(['heuristic:Common.text_edit']); + }); +}); + +// ── Reporting baseline (`since`) — tri-state contract ───────────────────── +// +// All reporting queries accept `opts.since`: +// - undefined ⇒ read store.getBaselineTs(); absent ⇒ no filter +// - null ⇒ explicit bypass (engine-state callers) +// - ISO ⇒ filter d.ts >= since (or pf.ts for fix-rule queries) +// +// Each test below seeds two timestamps — one before a midpoint, one after — +// and asserts the query honours the explicit ISO, the absent meta default +// (full history), and the meta-set value (auto-applied default). + +describe('reporting baseline: since param', () => { + const OLD = '2026-04-01T00:00:00.000Z'; // pre-midpoint + const NEW = '2026-04-30T00:00:00.000Z'; // post-midpoint + const MID = '2026-04-15T00:00:00.000Z'; + + function seedTwoEras() { + // Old + new emit on the same template — easy to count. Two emits in the + // OLD era share session 's1' so a single window can host both outcomes + // and the diagnostic↔outcome (fp, session_id, file) JOIN matches. + emitEvent(store, 's1', 'old1', 'CheckA', OLD); + emitEvent(store, 's1', 'old2', 'CheckA', OLD); + emitEvent(store, 's3', 'new1', 'CheckA', NEW); + } + + function seedTwoErasWithOutcomes() { + seedTwoEras(); + const wOld = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: OLD, ts_end: OLD, + }); + const wNew = store.insertWindow({ + session_id: 's3', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: NEW, ts_end: NEW, + }); + store.insertOutcome({ fp: 'old1', window_id: wOld, outcome: 'regressed' }); + store.insertOutcome({ fp: 'old2', window_id: wOld, outcome: 'regressed' }); + store.insertOutcome({ fp: 'new1', window_id: wNew, outcome: 'resolved' }); + } + + test('checkScorecards: ISO since filters out pre-baseline emits', () => { + seedTwoEras(); + const all = checkScorecards(store, { minCohort: 1 }); + expect(all[0].emitted).toBe(3); + const post = checkScorecards(store, { minCohort: 1, since: MID }); + expect(post[0].emitted).toBe(1); + }); + + test('checkScorecards: outcome counts honour the same baseline', () => { + seedTwoErasWithOutcomes(); + const post = checkScorecards(store, { minCohort: 1, since: MID }); + // Only the post-baseline emit's outcome (resolved) should count. + expect(post[0].sample_size).toBe(1); + expect(post[0].resolution_rate.mean).toBeGreaterThan(0.5); + }); + + test('checkScorecards: since=null bypasses meta baseline', () => { + seedTwoEras(); + store.setBaselineTs(MID); + const all = checkScorecards(store, { minCohort: 1, since: null }); + expect(all[0].emitted).toBe(3); + store.clearBaseline(); + }); + + test('checkScorecards: since=undefined reads meta baseline by default', () => { + seedTwoEras(); + store.setBaselineTs(MID); + const post = checkScorecards(store, { minCohort: 1 }); + expect(post[0].emitted).toBe(1); + store.clearBaseline(); + }); + + test('rulePerformance: ISO since filters by d.ts', () => { + seedTwoEras(); + const all = rulePerformance(store); + expect(all[0].emitted).toBe(3); + const post = rulePerformance(store, { since: MID }); + expect(post[0].emitted).toBe(1); + }); + + test('rulePerformance: outcome counts narrow to the window', () => { + seedTwoErasWithOutcomes(); + const post = rulePerformance(store, { since: MID }); + expect(post[0].total_outcomes).toBe(1); + expect(post[0].resolved).toBe(1); + expect(post[0].regressed).toBe(0); + }); + + test('rulePerformance: meta default fires when since is absent', () => { + seedTwoEras(); + store.setBaselineTs(MID); + expect(rulePerformance(store)[0].emitted).toBe(1); + store.clearBaseline(); + expect(rulePerformance(store)[0].emitted).toBe(3); + }); + + test('fixAdoptionFunnel: emit + outcome counts narrow to the window', () => { + seedTwoErasWithOutcomes(); + const post = fixAdoptionFunnel(store, { since: MID }); + expect(post.emitted).toBe(1); + expect(post.resolved).toBe(1); + expect(post.regressed).toBe(0); + const all = fixAdoptionFunnel(store); + expect(all.emitted).toBe(3); + expect(all.regressed).toBe(2); + }); + + test('fixAdoptionFunnel: meta baseline + since=null bypass', () => { + seedTwoErasWithOutcomes(); + store.setBaselineTs(MID); + expect(fixAdoptionFunnel(store).emitted).toBe(1); + expect(fixAdoptionFunnel(store, { since: null }).emitted).toBe(3); + store.clearBaseline(); + }); + + test('knowledgeGaps: filters total_emitted by since', () => { + // Need ≥3 emits to pass the HAVING gate post-filter. + emitEvent(store, 's1', 'old1', 'KGCheck', OLD); + emitEvent(store, 's1', 'old2', 'KGCheck', OLD); + emitEvent(store, 's1', 'new1', 'KGCheck', NEW); + emitEvent(store, 's1', 'new2', 'KGCheck', NEW); + emitEvent(store, 's1', 'new3', 'KGCheck', NEW); + + const all = knowledgeGaps(store); + const allRow = all.find(r => r.check === 'KGCheck'); + expect(allRow?.total_emitted).toBe(5); + + const post = knowledgeGaps(store, { since: MID }); + const postRow = post.find(r => r.check === 'KGCheck'); + expect(postRow?.total_emitted).toBe(3); + }); + + test('confidenceCalibration: filters by d.ts', () => { + const wid = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: OLD, ts_end: NEW, + }); + // Old: low confidence, regressed. New: high confidence, resolved. + store.ingestEvent({ + v: 1, session_id: 's1', ts: OLD, kind: 'validator_emit', + fp: 'cal-old', file: 'app/views/pages/index.html.liquid', + hint_rule_id: 'X', confidence: 0.2, proposed_fixes: [], + }); + store.ingestEvent({ + v: 1, session_id: 's1', ts: NEW, kind: 'validator_emit', + fp: 'cal-new', file: 'app/views/pages/index.html.liquid', + hint_rule_id: 'X', confidence: 0.9, proposed_fixes: [], + }); + store.insertOutcome({ fp: 'cal-old', window_id: wid, outcome: 'regressed' }); + store.insertOutcome({ fp: 'cal-new', window_id: wid, outcome: 'resolved' }); + + const allBuckets = confidenceCalibration(store); + const allTotal = allBuckets.reduce((s, b) => s + b.sample_size, 0); + expect(allTotal).toBe(2); + + const postBuckets = confidenceCalibration(store, { since: MID }); + const postTotal = postBuckets.reduce((s, b) => s + b.sample_size, 0); + expect(postTotal).toBe(1); + }); + + test('ruleScoresByCategory: filters by d.ts', () => { + seedTwoErasWithOutcomes(); + const all = ruleScoresByCategory(store); + const allRow = all.find(r => r.rule_id === 'CheckA'); + expect(allRow.outcomes).toBe(3); + const post = ruleScoresByCategory(store, { since: MID }); + const postRow = post.find(r => r.rule_id === 'CheckA'); + expect(postRow.outcomes).toBe(1); + }); + + test('sessionSummaries: filters session list to those active in window', () => { + // Distinct sessions per era. + store.ingestEvent({ v: 1, session_id: 'old-only', ts: OLD, kind: 'server_start', + project_dir: '/tmp', version: '1.0', started_at: OLD }); + toolCallEvent(store, 'old-only', 'validate_code', OLD); + toolCallEvent(store, 'new-only', 'validate_code', NEW); + + const all = sessionSummaries(store); + expect(all.map(s => s.session_id).sort()).toEqual(['new-only', 'old-only']); + + const post = sessionSummaries(store, { since: MID }); + expect(post.map(s => s.session_id)).toEqual(['new-only']); + }); + + test('toolSequenceBigrams: filters events by ts', () => { + toolCallEvent(store, 's1', 'project_map', OLD); + toolCallEvent(store, 's1', 'scaffold', OLD); + toolCallEvent(store, 's1', 'project_map', NEW); + toolCallEvent(store, 's1', 'validate_code', NEW); + + const all = toolSequenceBigrams(store); + expect(all.find(b => b.bigram[0] === 'project_map' && b.bigram[1] === 'scaffold')).toBeDefined(); + + const post = toolSequenceBigrams(store, { since: MID }); + expect(post.find(b => b.bigram[0] === 'project_map' && b.bigram[1] === 'scaffold')).toBeUndefined(); + expect(post.find(b => b.bigram[0] === 'project_map' && b.bigram[1] === 'validate_code')).toBeDefined(); + }); + + test('diagnosticJourney: filters timeline to post-baseline emits', () => { + store.ingestEvent({ + v: 1, session_id: 's1', ts: OLD, kind: 'validator_emit', + fp: 'fp-old', template_fp: 'jt', file: 'app/views/pages/x.liquid', + hint_rule_id: 'CheckJ', proposed_fixes: [], + }); + store.ingestEvent({ + v: 1, session_id: 's2', ts: NEW, kind: 'validator_emit', + fp: 'fp-new', template_fp: 'jt', file: 'app/views/pages/x.liquid', + hint_rule_id: 'CheckJ', proposed_fixes: [], + }); + + const all = diagnosticJourney(store, 'jt'); + expect(all.session_count).toBe(2); + const post = diagnosticJourney(store, 'jt', { since: MID }); + expect(post.session_count).toBe(1); + expect(post.timeline[0].session_id).toBe('s2'); + }); + + test('ruleDrilldown: filters samples + file/template stats', () => { + seedTwoErasWithOutcomes(); + const all = ruleDrilldown(store, 'CheckA'); + expect(all.samples).toHaveLength(3); + expect(all.file_distribution[0].emitted).toBe(3); + + const post = ruleDrilldown(store, 'CheckA', { since: MID }); + expect(post.samples).toHaveLength(1); + expect(post.file_distribution[0].emitted).toBe(1); + expect(post.file_distribution[0].resolved).toBe(1); + expect(post.file_distribution[0].regressed).toBe(0); + }); + + test('fixRulePerformance: filters by pf.ts', () => { + function emitWithFix(sid, fp, ts) { + store.ingestEvent({ + v: 1, session_id: sid, ts, kind: 'validator_emit', + fp, template_fp: 'tpl', file: 'app/views/pages/x.liquid', + hint_rule_id: 'X.r', + proposed_fixes: [{ range: null, new_text_hash: 'h', kind: 'text_edit', rule_id: 'X.r' }], + }); + } + emitWithFix('s1', 'oldA', OLD); + emitWithFix('s2', 'oldB', OLD); + emitWithFix('s3', 'new1', NEW); + + const all = fixRulePerformance(store); + expect(all.find(r => r.rule_id === 'X.r').proposed).toBe(3); + + const post = fixRulePerformance(store, { since: MID }); + expect(post.find(r => r.rule_id === 'X.r').proposed).toBe(1); + }); + + test('recommendations: forwards since to checkScorecards', () => { + // Old era: clearly harmful. New era: clean. + for (let i = 0; i < 10; i++) { + emitEvent(store, 's1', `bad-${i}`, 'BadCheck', OLD); + } + const wOld = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: OLD, ts_end: OLD, + }); + for (let i = 0; i < 10; i++) { + store.insertOutcome({ fp: `bad-${i}`, window_id: wOld, outcome: 'regressed' }); + } + + const allRecs = recommendations(store, 0.3); + expect(allRecs.find(r => r.check === 'BadCheck')).toBeDefined(); + const postRecs = recommendations(store, 0.3, { since: MID }); + expect(postRecs.find(r => r.check === 'BadCheck')).toBeUndefined(); + }); + + test('resolveSince precedence: explicit ISO beats meta baseline', () => { + // meta says MID, query says OLD → query wins, sees everything. + store.setBaselineTs(MID); + seedTwoEras(); + const out = checkScorecards(store, { minCohort: 1, since: OLD }); + expect(out[0].emitted).toBe(3); + store.clearBaseline(); + }); + + test('store without getBaselineTs (mock) — undefined since means no filter', () => { + // Defensive: resolveSince must degrade gracefully when given a partial mock. + const fakeStore = { + query: store.query, + queryOne: store.queryOne, + // Note: no getBaselineTs. + }; + seedTwoEras(); + // Bind the real prepared-statement methods to the real db path so the + // fake store can still issue queries (we just test the resolver path). + fakeStore.query = (sql, params) => store.query(sql, params); + fakeStore.queryOne = (sql, params) => store.queryOne(sql, params); + + const out = checkScorecards(fakeStore, { minCohort: 1 }); + expect(out[0].emitted).toBe(3); + }); +}); diff --git a/tests/unit/analytics-store.test.js b/tests/unit/analytics-store.test.js new file mode 100644 index 0000000..345e3e8 --- /dev/null +++ b/tests/unit/analytics-store.test.js @@ -0,0 +1,718 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { openAnalyticsStore } from '../../src/core/analytics-store.js'; +import { openBlobStore } from '../../src/core/blob-store.js'; +import { fingerprint, templateOf } from '../../src/core/diagnostic-record.js'; +import { mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +function tmpPath() { + return join(tmpdir(), `pos-analytics-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); +} + +function tmpSessionDir() { + const dir = join(tmpdir(), `pos-sessions-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}`); + mkdirSync(dir, { recursive: true }); + return dir; +} + +function makeValidatorEmitLine(overrides = {}) { + const event = { + v: 1, + session_id: 'test-session', + ts: '2026-04-17T10:00:00.000Z', + kind: 'validator_emit', + fp: 'abc123', + template_fp: 'tpl456', + file: 'app/views/pages/index.html.liquid', + hint_rule_id: 'MissingPartial.suggest_nearest', + content_hash: 'hash789', + proposed_fixes: [], + ...overrides, + }; + return JSON.stringify(event); +} + +function makeToolCallLine(overrides = {}) { + const event = { + v: 1, + session_id: 'test-session', + ts: '2026-04-17T10:00:01.000Z', + kind: 'tool_call', + tool: 'validate_code', + duration_ms: 150, + success: true, + ...overrides, + }; + return JSON.stringify(event); +} + +function makeServerStartLine(overrides = {}) { + const event = { + v: 1, + session_id: 'test-session', + ts: '2026-04-17T10:00:00.000Z', + kind: 'server_start', + project_dir: '/tmp/test', + version: '0.5.2', + started_at: '2026-04-17T10:00:00.000Z', + ...overrides, + }; + return JSON.stringify(event); +} + +let store; +let dbPath; + +beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); +}); + +afterEach(() => { + try { store.close(); } catch {} + try { rmSync(dbPath, { force: true }); } catch {} + try { rmSync(dbPath + '-wal', { force: true }); } catch {} + try { rmSync(dbPath + '-shm', { force: true }); } catch {} +}); + +describe('openAnalyticsStore', () => { + test('creates database with schema', () => { + const s = store.stats(); + expect(s.schema_version).toBe(6); + expect(s.events).toBe(0); + expect(s.diagnostics).toBe(0); + }); + + test('meta stores and retrieves values', () => { + expect(store.getMeta('schema_version')).toBe('6'); + expect(store.getMeta('nonexistent')).toBeNull(); + }); +}); + +describe('reporting baseline', () => { + test('absent by default — getBaselineTs returns null', () => { + expect(store.getBaselineTs()).toBeNull(); + const meta = store.getBaselineMeta(); + expect(meta.baseline_ts).toBeNull(); + expect(meta.set_at).toBeNull(); + }); + + test('setBaselineTs persists ISO + stamps set_at', () => { + const ts = '2026-04-30T12:00:00.000Z'; + store.setBaselineTs(ts); + expect(store.getBaselineTs()).toBe(ts); + const meta = store.getBaselineMeta(); + expect(meta.baseline_ts).toBe(ts); + // set_at is the wall-clock moment of the call — non-null + parseable. + expect(meta.set_at).not.toBeNull(); + expect(Number.isFinite(new Date(meta.set_at).getTime())).toBe(true); + }); + + test('setBaselineTs(null) clears both keys', () => { + store.setBaselineTs('2026-04-30T12:00:00.000Z'); + expect(store.getBaselineTs()).not.toBeNull(); + store.setBaselineTs(null); + expect(store.getBaselineTs()).toBeNull(); + expect(store.getBaselineMeta().set_at).toBeNull(); + }); + + test('clearBaseline() removes both keys', () => { + store.setBaselineTs('2026-04-30T12:00:00.000Z'); + store.clearBaseline(); + expect(store.getBaselineTs()).toBeNull(); + expect(store.getBaselineMeta().set_at).toBeNull(); + }); + + test('overwrites a prior baseline (single source of truth)', () => { + store.setBaselineTs('2026-04-01T00:00:00.000Z'); + store.setBaselineTs('2026-04-30T12:00:00.000Z'); + expect(store.getBaselineTs()).toBe('2026-04-30T12:00:00.000Z'); + // exactly one row each — no historical accumulation in the meta table + const rows = store.query( + `SELECT key FROM meta WHERE key IN ('analytics_baseline_ts','analytics_baseline_set_at')`, + ); + expect(rows).toHaveLength(2); + }); + + test('rejects non-ISO inputs without mutating state', () => { + store.setBaselineTs('2026-04-30T12:00:00.000Z'); + const before = store.getBaselineTs(); + expect(() => store.setBaselineTs('not-a-date')).toThrow(TypeError); + expect(() => store.setBaselineTs(123)).toThrow(TypeError); + expect(() => store.setBaselineTs('')).toThrow(TypeError); + expect(store.getBaselineTs()).toBe(before); + }); + + test('survives store close/reopen — persisted in meta table', () => { + const ts = '2026-04-30T12:00:00.000Z'; + store.setBaselineTs(ts); + store.close(); + store = openAnalyticsStore(dbPath); + expect(store.getBaselineTs()).toBe(ts); + }); + + test('rebuild() preserves the baseline (rebuild clears derived data, not meta)', () => { + const sessionsRoot = tmpSessionDir(); + const sess = join(sessionsRoot, 'session-1'); + mkdirSync(sess, { recursive: true }); + writeFileSync(join(sess, 'events.ndjson'), [ + makeServerStartLine({ session_id: 's1' }), + makeValidatorEmitLine({ session_id: 's1', fp: 'a' }), + ].join('\n') + '\n'); + + store.setBaselineTs('2026-04-30T12:00:00.000Z'); + store.rebuild(sessionsRoot); + expect(store.getBaselineTs()).toBe('2026-04-30T12:00:00.000Z'); + + rmSync(sessionsRoot, { recursive: true, force: true }); + }); +}); + +describe('ingestEvent', () => { + test('inserts generic event', () => { + store.ingestEvent({ + v: 1, session_id: 'sess1', ts: '2026-04-17T10:00:00Z', + kind: 'server_start', project_dir: '/tmp', version: '1.0', started_at: '2026-04-17T10:00:00Z', + }); + const s = store.stats(); + expect(s.events).toBe(1); + expect(s.diagnostics).toBe(0); + }); + + test('validator_emit populates diagnostics + proposed_fixes', () => { + store.ingestEvent({ + v: 1, session_id: 'sess1', ts: '2026-04-17T10:00:00Z', + kind: 'validator_emit', + fp: 'fp1', template_fp: 'tfp1', file: 'app/views/pages/index.html.liquid', + hint_rule_id: 'MissingPartial.suggest_nearest', + content_hash: 'ch1', + proposed_fixes: [ + { range: { start: { line: 1, character: 0 } }, new_text_hash: 'nth1', kind: 'replace' }, + ], + }); + const s = store.stats(); + expect(s.events).toBe(1); + expect(s.diagnostics).toBe(1); + + const diag = store.query('SELECT * FROM diagnostics WHERE fp = ?', ['fp1']); + expect(diag).toHaveLength(1); + expect(diag[0].check_name).toBe('MissingPartial.suggest_nearest'); + expect(diag[0].content_hash).toBe('ch1'); + + const fixes = store.query('SELECT * FROM proposed_fixes WHERE fp = ?', ['fp1']); + expect(fixes).toHaveLength(1); + expect(fixes[0].kind).toBe('replace'); + expect(fixes[0].new_text_hash).toBe('nth1'); + }); + + test('validator_emit without proposed_fixes still inserts diagnostic', () => { + store.ingestEvent({ + v: 1, session_id: 'sess1', ts: '2026-04-17T10:00:00Z', + kind: 'validator_emit', + fp: 'fp2', file: 'app/views/pages/about.html.liquid', + hint_rule_id: null, proposed_fixes: [], + }); + expect(store.stats().diagnostics).toBe(1); + const fixes = store.query('SELECT * FROM proposed_fixes WHERE fp = ?', ['fp2']); + expect(fixes).toHaveLength(0); + }); +}); + +describe('ingestSession', () => { + test('reads events.ndjson and populates database', () => { + const sessDir = tmpSessionDir(); + const lines = [ + makeServerStartLine(), + makeValidatorEmitLine({ fp: 'e1' }), + makeValidatorEmitLine({ fp: 'e2', ts: '2026-04-17T10:00:01.000Z' }), + makeToolCallLine(), + ].join('\n'); + writeFileSync(join(sessDir, 'events.ndjson'), lines + '\n'); + + const count = store.ingestSession(sessDir); + expect(count).toBe(4); + + const s = store.stats(); + expect(s.events).toBe(4); + expect(s.diagnostics).toBe(2); + expect(s.sessions).toBe(1); + + rmSync(sessDir, { recursive: true, force: true }); + }); + + test('returns 0 for missing events.ndjson', () => { + const sessDir = tmpSessionDir(); + expect(store.ingestSession(sessDir)).toBe(0); + rmSync(sessDir, { recursive: true, force: true }); + }); +}); + +describe('rebuild', () => { + test('clears and re-ingests all sessions', () => { + const sessionsRoot = tmpSessionDir(); + const sess1 = join(sessionsRoot, 'session-1'); + const sess2 = join(sessionsRoot, 'session-2'); + mkdirSync(sess1, { recursive: true }); + mkdirSync(sess2, { recursive: true }); + + writeFileSync(join(sess1, 'events.ndjson'), [ + makeServerStartLine({ session_id: 's1' }), + makeValidatorEmitLine({ session_id: 's1', fp: 'a' }), + ].join('\n') + '\n'); + + writeFileSync(join(sess2, 'events.ndjson'), [ + makeServerStartLine({ session_id: 's2' }), + makeValidatorEmitLine({ session_id: 's2', fp: 'b' }), + makeValidatorEmitLine({ session_id: 's2', fp: 'c', ts: '2026-04-17T11:00:00Z' }), + ].join('\n') + '\n'); + + const result = store.rebuild(sessionsRoot); + expect(result.sessions).toBe(2); + expect(result.events).toBe(5); + + const s = store.stats(); + expect(s.sessions).toBe(2); + expect(s.diagnostics).toBe(3); + + expect(store.getMeta('last_rebuild')).not.toBeNull(); + + // Rebuild again — should produce same counts (idempotent clear + re-ingest) + const result2 = store.rebuild(sessionsRoot); + expect(result2.sessions).toBe(2); + expect(store.stats().diagnostics).toBe(3); + + rmSync(sessionsRoot, { recursive: true, force: true }); + }); + + test('handles missing sessions dir', () => { + const result = store.rebuild('/tmp/nonexistent-analytics-test-dir-999'); + expect(result.sessions).toBe(0); + expect(result.events).toBe(0); + }); +}); + +describe('insertWindow + insertOutcome', () => { + test('inserts window and outcome rows', () => { + const windowId = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z', + content_hash_start: 'h1', content_hash_end: 'h2', + }); + expect(typeof windowId).toBe('number'); + + store.insertOutcome({ + fp: 'fp1', window_id: windowId, outcome: 'resolved', + fix_applied: 'verbatim', collateral_added: 0, + }); + + const s = store.stats(); + expect(s.windows).toBe(1); + expect(s.outcomes).toBe(1); + + const outcomes = store.query('SELECT * FROM outcomes WHERE fp = ?', ['fp1']); + expect(outcomes).toHaveLength(1); + expect(outcomes[0].outcome).toBe('resolved'); + expect(outcomes[0].fix_applied).toBe('verbatim'); + // Denormalized columns are derived from the window when not passed in. + expect(outcomes[0].session_id).toBe('s1'); + expect(outcomes[0].file).toBe('app/views/pages/index.html.liquid'); + }); + + test('dedupes on (session_id, file, fp) — terminal state wins', () => { + const w1 = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z', + }); + const w2 = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 1, + ts_start: '2026-04-17T10:01:00Z', ts_end: '2026-04-17T10:02:00Z', + }); + const w3 = store.insertWindow({ + session_id: 's1', file: 'app/views/pages/index.html.liquid', idx: 2, + ts_start: '2026-04-17T10:02:00Z', ts_end: '2026-04-17T10:03:00Z', + }); + + // Diagnostic flips: resolved in W1, regressed in W2, unchanged in W3. + store.insertOutcome({ fp: 'dup', window_id: w1, outcome: 'resolved' }); + store.insertOutcome({ fp: 'dup', window_id: w2, outcome: 'regressed' }); + store.insertOutcome({ fp: 'dup', window_id: w3, outcome: 'unchanged' }); + + const rows = store.query('SELECT * FROM outcomes WHERE fp = ?', ['dup']); + expect(rows).toHaveLength(1); + expect(rows[0].outcome).toBe('unchanged'); + expect(rows[0].window_id).toBe(w3); + }); + + test('different sessions with same fp stay separate', () => { + const wA = store.insertWindow({ + session_id: 'sA', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T10:00:00Z', ts_end: '2026-04-17T10:01:00Z', + }); + const wB = store.insertWindow({ + session_id: 'sB', file: 'app/views/pages/index.html.liquid', idx: 0, + ts_start: '2026-04-17T11:00:00Z', ts_end: '2026-04-17T11:01:00Z', + }); + store.insertOutcome({ fp: 'shared', window_id: wA, outcome: 'resolved' }); + store.insertOutcome({ fp: 'shared', window_id: wB, outcome: 'regressed' }); + + const rows = store.query('SELECT session_id, outcome FROM outcomes WHERE fp = ? ORDER BY session_id', ['shared']); + expect(rows).toHaveLength(2); + expect(rows[0].session_id).toBe('sA'); + expect(rows[0].outcome).toBe('resolved'); + expect(rows[1].session_id).toBe('sB'); + expect(rows[1].outcome).toBe('regressed'); + }); +}); + +describe('query helpers', () => { + test('query returns all matching rows', () => { + store.ingestEvent({ + v: 1, session_id: 's1', ts: '2026-04-17T10:00:00Z', + kind: 'validator_emit', fp: 'x1', file: 'f1.liquid', + hint_rule_id: 'check_a', proposed_fixes: [], + }); + store.ingestEvent({ + v: 1, session_id: 's1', ts: '2026-04-17T10:00:01Z', + kind: 'validator_emit', fp: 'x2', file: 'f2.liquid', + hint_rule_id: 'check_a', proposed_fixes: [], + }); + const rows = store.query('SELECT * FROM diagnostics WHERE check_name = ?', ['check_a']); + expect(rows).toHaveLength(2); + }); + + test('queryOne returns single row or null', () => { + store.ingestEvent({ + v: 1, session_id: 's1', ts: '2026-04-17T10:00:00Z', + kind: 'validator_emit', fp: 'only1', file: 'f.liquid', + hint_rule_id: 'c1', proposed_fixes: [], + }); + const row = store.queryOne('SELECT * FROM diagnostics WHERE fp = ?', ['only1']); + expect(row).not.toBeNull(); + expect(row.fp).toBe('only1'); + + const none = store.queryOne('SELECT * FROM diagnostics WHERE fp = ?', ['nope']); + expect(none).toBeNull(); + }); +}); + +describe('outcome dedup invariants', () => { + test('one outcome per (session, file, fp) across N flip windows (terminal state wins)', () => { + const sessDir = tmpSessionDir(); + const diag = { check: 'MissingPartial', message: "Missing partial 'blog/card'", severity: 'error', line: 1 }; + // Diagnostic flips present/absent four times across five validate_code calls. + // Pre-A1 this yielded four outcome rows for the same fp (alternating + // resolved/regressed). Post-A1 there is exactly one row and it reflects + // the terminal state — the last window saw the diagnostic reappear. + const calls = [ + { ts: '2026-04-17T10:00:00.000Z', output: { errors: [diag], warnings: [] } }, + { ts: '2026-04-17T10:01:00.000Z', output: { errors: [], warnings: [] } }, + { ts: '2026-04-17T10:02:00.000Z', output: { errors: [diag], warnings: [] } }, + { ts: '2026-04-17T10:03:00.000Z', output: { errors: [], warnings: [] } }, + { ts: '2026-04-17T10:04:00.000Z', output: { errors: [diag], warnings: [] } }, + ].map(c => JSON.stringify({ + v: 1, session_id: 'test-session', kind: 'tool_call', tool: 'validate_code', + duration_ms: 80, success: true, + input: { file_path: 'app/views/pages/index.html.liquid', content: '{% render "blog/card" %}' }, + ts: c.ts, output: c.output, + })); + const lines = [makeServerStartLine(), ...calls].join('\n'); + writeFileSync(join(sessDir, 'events.ndjson'), lines + '\n'); + + store.ingestSession(sessDir); + + const outcomeRows = store.query(`SELECT outcome FROM outcomes`); + expect(outcomeRows).toHaveLength(1); + expect(outcomeRows[0].outcome).toBe('regressed'); + + // Invariant: resolved ≤ distinct fps with outcomes. Here resolved = 0. + const resolved = store.queryOne(`SELECT COUNT(*) AS c FROM outcomes WHERE outcome = 'resolved'`).c; + const uniqueFps = store.queryOne(`SELECT COUNT(DISTINCT fp) AS c FROM outcomes`).c; + expect(resolved).toBeLessThanOrEqual(uniqueFps); + + rmSync(sessDir, { recursive: true, force: true }); + }); +}); + +describe('window classification via ingestion', () => { + test('produces windows and outcomes from consecutive validate_code calls', () => { + const sessDir = tmpSessionDir(); + const lines = [ + makeServerStartLine(), + JSON.stringify({ + v: 1, session_id: 'test-session', ts: '2026-04-17T10:00:01.000Z', + kind: 'tool_call', tool: 'validate_code', duration_ms: 100, success: true, + input: { file_path: 'app/views/pages/index.html.liquid', content: '{% render "blog_posts/card" %}' }, + output: { errors: [{ check: 'MissingPartial', message: "Missing partial 'blog_posts/card'", severity: 'error', line: 1 }], warnings: [] }, + }), + JSON.stringify({ + v: 1, session_id: 'test-session', ts: '2026-04-17T10:02:00.000Z', + kind: 'tool_call', tool: 'validate_code', duration_ms: 80, success: true, + input: { file_path: 'app/views/pages/index.html.liquid', content: '{% render "blog_posts/card" %}' }, + output: { errors: [], warnings: [] }, + }), + ].join('\n'); + writeFileSync(join(sessDir, 'events.ndjson'), lines + '\n'); + + store.ingestSession(sessDir); + const s = store.stats(); + expect(s.windows).toBe(1); + expect(s.outcomes).toBe(1); + + const outcomes = store.query('SELECT * FROM outcomes'); + expect(outcomes[0].outcome).toBe('resolved'); + + rmSync(sessDir, { recursive: true, force: true }); + }); +}); + +describe('real fixture ingestion', () => { + test('ingests fixture sessions without error', () => { + const fixtureDir = join(import.meta.dir, '../../tests/fixtures/broken-project/.pos-supervisor/sessions'); + const result = store.rebuild(fixtureDir); + expect(result.sessions).toBeGreaterThan(0); + expect(result.events).toBeGreaterThan(0); + + const s = store.stats(); + expect(s.diagnostics).toBeGreaterThan(0); + }); +}); + +// ── proposed_fixes.rule_id persistence (I1) ───────────────────────────────── + +describe('proposed_fixes rule_id persistence', () => { + test('ingestValidatorEmit writes rule_id on every fix', () => { + store.ingestEvent({ + v: 1, session_id: 's1', ts: '2026-04-17T10:00:00Z', + kind: 'validator_emit', + fp: 'fp-i1', template_fp: 'tpl', file: 'app/views/pages/x.liquid', + proposed_fixes: [ + { range: null, new_text_hash: 'h1', kind: 'text_edit', rule_id: 'UnknownFilter.suggest_nearest' }, + { range: null, new_text_hash: 'h2', kind: 'text_edit', rule_id: 'heuristic:UnknownFilter.text_edit' }, + { range: null, new_text_hash: null, kind: 'guidance', rule_id: null }, + ], + }); + const rows = store.query(`SELECT kind, rule_id FROM proposed_fixes WHERE fp = 'fp-i1' ORDER BY rowid`); + expect(rows).toHaveLength(3); + expect(rows[0].rule_id).toBe('UnknownFilter.suggest_nearest'); + expect(rows[1].rule_id).toBe('heuristic:UnknownFilter.text_edit'); + expect(rows[2].rule_id).toBeNull(); + }); +}); + +// ── hint_md_hash persistence (Bug 2) ───────────────────────────────────────── + +describe('hint_md_hash persistence', () => { + test('ingestValidatorEmit writes hint_md_hash column', () => { + store.ingestEvent({ + v: 1, + session_id: 'hs', + ts: '2026-04-17T10:00:00.000Z', + kind: 'validator_emit', + fp: 'hfp', + template_fp: 'htpl', + file: 'app/views/pages/x.liquid', + hint_rule_id: 'X.rule', + hint_md_hash: 'deadbeef'.repeat(8), + content_hash: 'feedface'.repeat(8), + proposed_fixes: [], + }); + const row = store.queryOne( + `SELECT hint_md_hash FROM diagnostics WHERE fp = 'hfp'`, + ); + expect(row.hint_md_hash).toBe('deadbeef'.repeat(8)); + }); + + test('null hint_md_hash stays null', () => { + store.ingestEvent({ + v: 1, + session_id: 'hs2', + ts: '2026-04-17T10:00:00.000Z', + kind: 'validator_emit', + fp: 'hfp2', + file: 'app/views/pages/y.liquid', + content_hash: 'feedface'.repeat(8), + proposed_fixes: [], + }); + const row = store.queryOne( + `SELECT hint_md_hash FROM diagnostics WHERE fp = 'hfp2'`, + ); + expect(row.hint_md_hash).toBeNull(); + }); +}); + +// ── fix_applied classification via blobStore (Bug 1) ───────────────────────── + +describe('classifyAndStoreWindows: fix_applied classification', () => { + let blobDir; + let blobStore; + beforeEach(() => { + blobDir = join(tmpdir(), `pos-blobs-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}`); + blobStore = openBlobStore(blobDir); + }); + afterEach(() => { + try { rmSync(blobDir, { recursive: true, force: true }); } catch {} + }); + + /** + * Write a session whose validator_emit fp matches the window classifier's + * computed fp. Both paths use fingerprint(check, file, messageTemplate) — + * diverging breaks the emit-index lookup. Helper shared by the test cases + * below so the fp calculation is visible in one place. + */ + function fpRow(sessDir, { sid, file, startContent, endContent, fixText, startTs, endTs, startDiag, endDiag }) { + const startHash = blobStore.put(startContent); + blobStore.put(endContent); + const fixHash = blobStore.put(fixText); + + const tmpl = templateOf(startDiag.check, startDiag.message); + const fp = fingerprint(startDiag.check, file, tmpl); + + const lines = [ + makeServerStartLine({ session_id: sid }), + JSON.stringify({ + v: 1, session_id: sid, ts: startTs, + kind: 'validator_emit', + fp, template_fp: 'tpl', file, + content_hash: startHash, + proposed_fixes: [{ range: null, new_text_hash: fixHash, kind: 'text_edit' }], + }), + JSON.stringify({ + v: 1, session_id: sid, ts: startTs, + kind: 'tool_call', tool: 'validate_code', duration_ms: 50, success: true, + input: { file_path: file, content: startContent }, + output: { errors: [startDiag], warnings: [] }, + }), + JSON.stringify({ + v: 1, session_id: sid, ts: endTs, + kind: 'tool_call', tool: 'validate_code', duration_ms: 45, success: true, + input: { file_path: file, content: endContent }, + output: { errors: endDiag ? [endDiag] : [], warnings: [] }, + }), + ].join('\n'); + writeFileSync(join(sessDir, 'events.ndjson'), lines + '\n'); + return { fp }; + } + + test('resolved + fix text present in end content → verbatim', () => { + const sessDir = tmpSessionDir(); + store.close(); + store = openAnalyticsStore(dbPath, { blobStore }); + + fpRow(sessDir, { + sid: 'sfa', file: 'app/views/pages/x.liquid', + startContent: "{{ 'hello' | capitalise }}", + endContent: "{{ 'hello' | capitalize }}", + fixText: 'capitalize', + startTs: '2026-04-17T10:00:01.000Z', endTs: '2026-04-17T10:01:00.000Z', + startDiag: { check: 'UnknownFilter', message: "Unknown filter 'capitalise'", severity: 'error', line: 1 }, + }); + + store.ingestSession(sessDir); + + const rows = store.query(`SELECT outcome, fix_applied FROM outcomes`); + expect(rows).toHaveLength(1); + expect(rows[0].outcome).toBe('resolved'); + expect(rows[0].fix_applied).toBe('verbatim'); + + rmSync(sessDir, { recursive: true, force: true }); + }); + + test('resolved + fix text NOT in end content → partial', () => { + const sessDir = tmpSessionDir(); + store.close(); + store = openAnalyticsStore(dbPath, { blobStore }); + + fpRow(sessDir, { + sid: 'sfa2', file: 'app/views/pages/y.liquid', + startContent: "{{ 'hello' | capitalise }}", + endContent: "{{ 'hello' | upcase }}", + fixText: 'capitalize', + startTs: '2026-04-17T10:00:01.000Z', endTs: '2026-04-17T10:01:00.000Z', + startDiag: { check: 'UnknownFilter', message: "Unknown filter 'capitalise'", severity: 'error', line: 1 }, + }); + + store.ingestSession(sessDir); + + const rows = store.query(`SELECT outcome, fix_applied FROM outcomes`); + expect(rows[0].outcome).toBe('resolved'); + expect(rows[0].fix_applied).toBe('partial'); + + rmSync(sessDir, { recursive: true, force: true }); + }); + + test('no blobStore → fix_applied stays null (backward compat)', () => { + const sessDir = tmpSessionDir(); + const lines = [ + makeServerStartLine(), + JSON.stringify({ + v: 1, session_id: 'test-session', ts: '2026-04-17T10:00:01.000Z', + kind: 'tool_call', tool: 'validate_code', duration_ms: 100, success: true, + input: { file_path: 'app/views/pages/z.liquid', content: "X" }, + output: { errors: [{ check: 'UnknownFilter', message: "Unknown filter 'x'", severity: 'error', line: 1 }], warnings: [] }, + }), + JSON.stringify({ + v: 1, session_id: 'test-session', ts: '2026-04-17T10:02:00.000Z', + kind: 'tool_call', tool: 'validate_code', duration_ms: 80, success: true, + input: { file_path: 'app/views/pages/z.liquid', content: "Y" }, + output: { errors: [], warnings: [] }, + }), + ].join('\n'); + writeFileSync(join(sessDir, 'events.ndjson'), lines + '\n'); + + store.ingestSession(sessDir); + const rows = store.query(`SELECT outcome, fix_applied FROM outcomes`); + expect(rows[0].outcome).toBe('resolved'); + expect(rows[0].fix_applied).toBeNull(); + + rmSync(sessDir, { recursive: true, force: true }); + }); + + test('regressed outcomes skip fix_applied classification', () => { + const sessDir = tmpSessionDir(); + store.close(); + store = openAnalyticsStore(dbPath, { blobStore }); + + const START = 'OLD'; + const END = 'NEW'; + const regressedDiag = { check: 'UnusedAssign', message: "Variable 'x' is assigned but never used", severity: 'warning', line: 1 }; + const file = 'app/views/pages/r.liquid'; + const fp = fingerprint(regressedDiag.check, file, templateOf(regressedDiag.check, regressedDiag.message)); + + const startHash = blobStore.put(START); + blobStore.put(END); + const fixHash = blobStore.put('IRRELEVANT'); + + const lines = [ + makeServerStartLine({ session_id: 'sr' }), + JSON.stringify({ + v: 1, session_id: 'sr', ts: '2026-04-17T10:01:00.000Z', + kind: 'validator_emit', + fp, template_fp: 'tpl', file, + content_hash: startHash, + proposed_fixes: [{ range: null, new_text_hash: fixHash, kind: 'text_edit' }], + }), + JSON.stringify({ + v: 1, session_id: 'sr', ts: '2026-04-17T10:00:00.000Z', + kind: 'tool_call', tool: 'validate_code', duration_ms: 50, success: true, + input: { file_path: file, content: START }, + output: { errors: [], warnings: [] }, + }), + JSON.stringify({ + v: 1, session_id: 'sr', ts: '2026-04-17T10:01:00.000Z', + kind: 'tool_call', tool: 'validate_code', duration_ms: 50, success: true, + input: { file_path: file, content: END }, + output: { errors: [regressedDiag], warnings: [] }, + }), + ].join('\n'); + writeFileSync(join(sessDir, 'events.ndjson'), lines + '\n'); + + store.ingestSession(sessDir); + + const rows = store.query(`SELECT outcome, fix_applied FROM outcomes WHERE fp = ?`, [fp]); + expect(rows[0].outcome).toBe('regressed'); + expect(rows[0].fix_applied).toBeNull(); + + rmSync(sessDir, { recursive: true, force: true }); + }); +}); diff --git a/tests/unit/analyze-project-diff.test.js b/tests/unit/analyze-project-diff.test.js index 33dc4ee..867b3c0 100644 --- a/tests/unit/analyze-project-diff.test.js +++ b/tests/unit/analyze-project-diff.test.js @@ -21,6 +21,7 @@ describe('computeBlockingFiles', () => { expect(result[0].lint_errors).toBe(2); expect(result[0].integrity_errors).toBe(0); expect(result[0].total).toBe(2); + expect(result[0].checks).toEqual([]); }); it('includes files with integrity errors only', () => { @@ -32,6 +33,7 @@ describe('computeBlockingFiles', () => { expect(result[0].path).toBe('app/views/partials/x.liquid'); expect(result[0].lint_errors).toBe(0); expect(result[0].integrity_errors).toBe(1); + expect(result[0].checks).toEqual(['broken_render']); }); it('merges lint and integrity errors for the same file', () => { @@ -47,6 +49,8 @@ describe('computeBlockingFiles', () => { expect(result[0].lint_errors).toBe(1); expect(result[0].integrity_errors).toBe(2); expect(result[0].total).toBe(3); + expect(result[0].checks).toContain('missing_graphql'); + expect(result[0].checks).toContain('broken_render'); }); it('ignores integrity warnings', () => { @@ -72,6 +76,46 @@ describe('computeBlockingFiles', () => { ]; expect(computeBlockingFiles(fileResults, [])).toHaveLength(0); }); + + it('extracts check names from allResults when provided', () => { + const fileResults = [ + { path: 'app/views/layouts/application.liquid', errors: 1, warnings: 0 }, + ]; + const allResults = { + errors: [ + { severity: 'error', _filePath: '/project/app/views/layouts/application.liquid', check: 'MetadataParamsCheck', message: 'Required parameter clear must be passed' }, + ], + }; + const result = computeBlockingFiles(fileResults, [], allResults); + expect(result).toHaveLength(1); + expect(result[0].checks).toEqual(['MetadataParamsCheck']); + }); + + it('collects multiple distinct check names per file', () => { + const fileResults = [ + { path: 'app/views/pages/broken.liquid', errors: 3, warnings: 0 }, + ]; + const allResults = { + errors: [ + { severity: 'error', _filePath: '/p/app/views/pages/broken.liquid', check: 'MetadataParamsCheck', message: 'a' }, + { severity: 'error', _filePath: '/p/app/views/pages/broken.liquid', check: 'MetadataParamsCheck', message: 'b' }, + { severity: 'error', _filePath: '/p/app/views/pages/broken.liquid', check: 'SyntaxError', message: 'c' }, + ], + }; + const result = computeBlockingFiles(fileResults, [], allResults); + expect(result[0].checks).toHaveLength(2); + expect(result[0].checks).toContain('MetadataParamsCheck'); + expect(result[0].checks).toContain('SyntaxError'); + }); + + it('works without allResults (backward compat)', () => { + const fileResults = [ + { path: 'app/a.liquid', errors: 1, warnings: 0 }, + ]; + const result = computeBlockingFiles(fileResults, []); + expect(result).toHaveLength(1); + expect(result[0].checks).toEqual([]); + }); }); // --------------------------------------------------------------------------- diff --git a/tests/unit/analyze-project.test.js b/tests/unit/analyze-project.test.js new file mode 100644 index 0000000..d775048 --- /dev/null +++ b/tests/unit/analyze-project.test.js @@ -0,0 +1,269 @@ +import { describe, it, expect } from 'bun:test'; +import { + getTranslationFilePaths, + filesAffectedByTranslationFile, + buildFixOrder, + computeBlockingFiles, +} from '../../src/tools/analyze-project.js'; + +const DIR = '/project'; + +// --------------------------------------------------------------------------- +// getTranslationFilePaths +// --------------------------------------------------------------------------- + +describe('getTranslationFilePaths', () => { + it('returns empty array when no translations in projectMap', () => { + expect(getTranslationFilePaths({})).toEqual([]); + expect(getTranslationFilePaths({ translations: {} })).toEqual([]); + }); + + it('maps locale keys to app/translations/.yml paths', () => { + const projectMap = { translations: { en: {}, de: {}, fr: {} } }; + const result = getTranslationFilePaths(projectMap); + expect(result).toHaveLength(3); + expect(result).toContain('app/translations/en.yml'); + expect(result).toContain('app/translations/de.yml'); + expect(result).toContain('app/translations/fr.yml'); + }); + + it('handles single locale', () => { + const result = getTranslationFilePaths({ translations: { en: {} } }); + expect(result).toEqual(['app/translations/en.yml']); + }); +}); + +// --------------------------------------------------------------------------- +// filesAffectedByTranslationFile +// --------------------------------------------------------------------------- + +describe('filesAffectedByTranslationFile', () => { + it('returns empty set for unknown locale path', () => { + const result = filesAffectedByTranslationFile('app/translations/.yml', {}); + expect(result.size).toBe(0); + }); + + it('returns empty set when no files use translation keys', () => { + const projectMap = { + pages: { show: { path: 'app/views/pages/show.html.liquid', translation_keys: [] } }, + partials: {}, + }; + const result = filesAffectedByTranslationFile('app/translations/en.yml', projectMap); + expect(result.size).toBe(0); + }); + + it('includes files that have translation_keys', () => { + const projectMap = { + pages: { + show: { path: 'app/views/pages/show.html.liquid', translation_keys: ['app.hello'] }, + }, + partials: { + header: { path: 'app/views/partials/header.liquid', translation_keys: ['app.nav.home'] }, + footer: { path: 'app/views/partials/footer.liquid', translation_keys: [] }, + }, + commands: {}, + queries: {}, + }; + const result = filesAffectedByTranslationFile('app/translations/en.yml', projectMap); + expect(result.has('app/views/pages/show.html.liquid')).toBe(true); + expect(result.has('app/views/partials/header.liquid')).toBe(true); + expect(result.has('app/views/partials/footer.liquid')).toBe(false); + }); + + it('includes commands and queries that use translation keys', () => { + const projectMap = { + pages: {}, + partials: {}, + commands: { + 'app/lib/commands/create.liquid': { + path: 'app/lib/commands/create.liquid', + translation_keys: ['app.success'], + }, + }, + queries: { + 'app/lib/queries/search.liquid': { + path: 'app/lib/queries/search.liquid', + translation_keys: ['app.no_results'], + }, + }, + }; + const result = filesAffectedByTranslationFile('app/translations/en.yml', projectMap); + expect(result.has('app/lib/commands/create.liquid')).toBe(true); + expect(result.has('app/lib/queries/search.liquid')).toBe(true); + }); + + it('works for non-en locales', () => { + const projectMap = { + pages: { show: { path: 'app/views/pages/show.html.liquid', translation_keys: ['app.hello'] } }, + partials: {}, + commands: {}, + queries: {}, + }; + const result = filesAffectedByTranslationFile('app/translations/de.yml', projectMap); + expect(result.has('app/views/pages/show.html.liquid')).toBe(true); + }); +}); + +// --------------------------------------------------------------------------- +// buildFixOrder — translation file scenarios +// --------------------------------------------------------------------------- + +function file(path, errors = 1, warnings = 0) { + return { path, errors, warnings }; +} + +describe('buildFixOrder — translation files', () => { + it('includes translation file in fix_order output', () => { + const files = [ + file('app/translations/en.yml'), + file('app/views/pages/show.html.liquid'), + ]; + const projectMap = { + pages: { show: { path: 'app/views/pages/show.html.liquid', translation_keys: ['app.hello'] } }, + partials: {}, + commands: {}, + queries: {}, + }; + const result = buildFixOrder(files, {}, DIR, projectMap); + expect(result).toHaveLength(2); + expect(result.map(r => r.path)).toContain('app/translations/en.yml'); + }); + + it('places translation file before dependent liquid files (via translation_keys)', () => { + const files = [ + file('app/views/pages/show.html.liquid'), + file('app/translations/en.yml'), + ]; + const projectMap = { + pages: { show: { path: 'app/views/pages/show.html.liquid', translation_keys: ['app.hello'] } }, + partials: {}, + commands: {}, + queries: {}, + }; + const result = buildFixOrder(files, {}, DIR, projectMap); + const paths = result.map(r => r.path); + expect(paths.indexOf('app/translations/en.yml')) + .toBeLessThan(paths.indexOf('app/views/pages/show.html.liquid')); + }); + + it('places translation file first when dependency graph has injected TranslationKeyExists edges', () => { + // Simulates what analyze_project does: inject edges for files with + // TranslationKeyExists errors so buildFixOrder can place translation file first. + const files = [ + file('app/views/pages/show.html.liquid'), + file('app/translations/en.yml'), + ]; + const depGraph = { + 'app/views/pages/show.html.liquid': { + depends_on: ['app/translations/en.yml'], + referenced_by: [], + }, + 'app/translations/en.yml': { + depends_on: [], + referenced_by: ['app/views/pages/show.html.liquid'], + }, + }; + const result = buildFixOrder(files, depGraph, DIR, {}); + expect(result[0].path).toBe('app/translations/en.yml'); + expect(result[0].dependents_with_errors).toBe(1); + }); + + it('counts dependent files with errors in dependents_with_errors', () => { + const files = [ + file('app/translations/en.yml'), + file('app/views/pages/a.html.liquid'), + file('app/views/pages/b.html.liquid'), + ]; + const projectMap = { + pages: { + a: { path: 'app/views/pages/a.html.liquid', translation_keys: ['app.x'] }, + b: { path: 'app/views/pages/b.html.liquid', translation_keys: ['app.y'] }, + }, + partials: {}, + commands: {}, + queries: {}, + }; + const result = buildFixOrder(files, {}, DIR, projectMap); + const tran = result.find(r => r.path === 'app/translations/en.yml'); + expect(tran.dependents_with_errors).toBe(2); + expect(tran.reason).toMatch(/Fix first/); + }); + + it('does not include translation dependents without errors', () => { + // Only translation file has errors; pages have no errors → not in fileResults + const files = [file('app/translations/en.yml')]; + const projectMap = { + pages: { show: { path: 'app/views/pages/show.html.liquid', translation_keys: ['app.hello'] } }, + partials: {}, + commands: {}, + queries: {}, + }; + const result = buildFixOrder(files, {}, DIR, projectMap); + expect(result).toHaveLength(1); + const tran = result[0]; + expect(tran.dependents_with_errors).toBe(0); + expect(tran.reason).toBe('No cross-error dependencies'); + }); + + it('works when projectMap is undefined (backward compat)', () => { + const files = [file('app/a.liquid')]; + expect(() => buildFixOrder(files, {}, DIR, undefined)).not.toThrow(); + const result = buildFixOrder(files, {}, DIR, undefined); + expect(result).toHaveLength(1); + }); +}); + +// --------------------------------------------------------------------------- +// computeBlockingFiles — translation file explicit coverage (Change 1b) +// --------------------------------------------------------------------------- + +describe('computeBlockingFiles — translation files', () => { + const projectMap = { translations: { en: {} } }; + + it('picks up translation file errors even when not in fileResults', () => { + // Simulate: user passed explicit files list without translation files. + // Translation error still in allResults from pos-cli check. + const allResults = { + errors: [ + { + severity: 'error', + _filePath: `${DIR}/app/translations/en.yml`, + check: 'MatchingTranslations', + message: 'Missing key "app.hello" in en', + }, + ], + }; + const result = computeBlockingFiles([], [], allResults, DIR, projectMap); + expect(result).toHaveLength(1); + expect(result[0].path).toBe('app/translations/en.yml'); + expect(result[0].lint_errors).toBe(1); + expect(result[0].checks).toContain('MatchingTranslations'); + }); + + it('does not duplicate translation file already in fileResults', () => { + // fileResults already has it (default discovery path with Change 1a) + const fileResults = [{ path: 'app/translations/en.yml', errors: 2, warnings: 0 }]; + const allResults = { + errors: [ + { severity: 'error', _filePath: `${DIR}/app/translations/en.yml`, check: 'MatchingTranslations', message: 'x' }, + ], + }; + const result = computeBlockingFiles(fileResults, [], allResults, DIR, projectMap); + const entries = result.filter(r => r.path === 'app/translations/en.yml'); + expect(entries).toHaveLength(1); + expect(entries[0].lint_errors).toBe(2); // from fileResults, not allResults + }); + + it('skips translation file when it has no errors in allResults', () => { + const allResults = { errors: [] }; + const result = computeBlockingFiles([], [], allResults, DIR, projectMap); + expect(result).toHaveLength(0); + }); + + it('works without projectDir/projectMap (backward compat)', () => { + const fileResults = [{ path: 'app/a.liquid', errors: 1, warnings: 0 }]; + expect(() => computeBlockingFiles(fileResults, [])).not.toThrow(); + const result = computeBlockingFiles(fileResults, []); + expect(result).toHaveLength(1); + }); +}); diff --git a/tests/unit/blob-store.test.js b/tests/unit/blob-store.test.js new file mode 100644 index 0000000..55896a5 --- /dev/null +++ b/tests/unit/blob-store.test.js @@ -0,0 +1,206 @@ +/** + * blob-store unit tests — pin the content-addressed write/read contract, + * idempotency, atomicity (no partial files visible mid-write), and LRU + * eviction under both byte and file-count caps. + */ + +import { describe, it, expect } from 'bun:test'; +import { mkdtempSync, rmSync, existsSync, readdirSync, writeFileSync, utimesSync, statSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { + BLOB_HASH_ALGO, + blobHash, + openBlobStore, +} from '../../src/core/blob-store.js'; + +function workDir() { + const dir = mkdtempSync(join(tmpdir(), 'pos-blobs-')); + return { dir, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; +} + +describe('blob-store: hashing', () => { + it('uses sha256', () => { + expect(BLOB_HASH_ALGO).toBe('sha256'); + }); + + it('blobHash matches a known sha256', () => { + // sha256('hello') = 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 + expect(blobHash('hello')).toBe('2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824'); + }); + + it('hashes Buffer and string the same when bytes match', () => { + expect(blobHash(Buffer.from('hello', 'utf-8'))).toBe(blobHash('hello')); + }); +}); + +describe('blob-store: put / get / exists', () => { + it('writes content under a sharded path and reads it back', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir); + const hash = store.put('the rain in spain'); + expect(hash).toBe(blobHash('the rain in spain')); + expect(store.exists(hash)).toBe(true); + + const got = store.getText(hash); + expect(got).toBe('the rain in spain'); + + // Path layout: /// + const expected = join(dir, hash.slice(0, 2), hash.slice(2, 4), hash.slice(4)); + expect(existsSync(expected)).toBe(true); + } finally { cleanup(); } + }); + + it('put is idempotent for the same content', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir); + const a = store.put('same'); + const b = store.put('same'); + expect(a).toBe(b); + expect(store.stats().count).toBe(1); + } finally { cleanup(); } + }); + + it('different content produces different hashes', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir); + expect(store.put('a')).not.toBe(store.put('b')); + expect(store.stats().count).toBe(2); + } finally { cleanup(); } + }); + + it('get / getText return null on miss', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir); + // Use a real-shaped sha256 hex so pathFor doesn't reject on shape. + const fakeHash = '0'.repeat(64); + expect(store.get(fakeHash)).toBeNull(); + expect(store.getText(fakeHash)).toBeNull(); + } finally { cleanup(); } + }); + + it('rejects malformed hashes in pathFor', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir); + expect(() => store.exists('xx')).toThrow(/hash/i); + expect(() => store.exists(null)).toThrow(/hash/i); + } finally { cleanup(); } + }); + + it('does not leak .tmp files into listEntries', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir); + const hash = store.put('staged'); + // Drop a synthetic temp file in the same shard dir. + const shard = join(dir, hash.slice(0, 2), hash.slice(2, 4)); + writeFileSync(join(shard, 'whatever.tmp.999.x'), 'partial'); + const hashes = store.listEntries().map((e) => e.hash); + expect(hashes).toEqual([hash]); + } finally { cleanup(); } + }); +}); + +describe('blob-store: remove', () => { + it('returns true on hit, false on miss', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir); + const hash = store.put('to-remove'); + expect(store.remove(hash)).toBe(true); + expect(store.exists(hash)).toBe(false); + expect(store.remove(hash)).toBe(false); + } finally { cleanup(); } + }); +}); + +describe('blob-store: enforceLimits (LRU)', () => { + it('evicts oldest by atime when maxFiles is exceeded', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir, { maxFiles: 2, maxBytes: 1024 }); + const hashA = store.put('a'); ageBlob(store.pathFor(hashA), 1000); + const hashB = store.put('b'); ageBlob(store.pathFor(hashB), 500); + const hashC = store.put('c'); // Should evict hashA (oldest atime). + + expect(store.exists(hashA)).toBe(false); + expect(store.exists(hashB)).toBe(true); + expect(store.exists(hashC)).toBe(true); + expect(store.stats().count).toBe(2); + } finally { cleanup(); } + }); + + it('evicts oldest by atime when maxBytes is exceeded', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir, { maxFiles: 100, maxBytes: 6 }); + const hashA = store.put('aaaa'); ageBlob(store.pathFor(hashA), 1000); // 4 bytes + const hashB = store.put('bbbb'); // 4 bytes — total 8, over cap → evict A + expect(store.exists(hashA)).toBe(false); + expect(store.exists(hashB)).toBe(true); + } finally { cleanup(); } + }); + + it('idempotent put bumps atime so repeated reads keep a blob warm', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir, { maxFiles: 2 }); + const hashA = store.put('a'); ageBlob(store.pathFor(hashA), 5000); + const hashB = store.put('b'); ageBlob(store.pathFor(hashB), 4000); + + // Re-put A — it should be the freshest; the next eviction should + // drop B instead of A. + store.put('a'); + const hashC = store.put('c'); + + expect(store.exists(hashA)).toBe(true); + expect(store.exists(hashB)).toBe(false); + expect(store.exists(hashC)).toBe(true); + } finally { cleanup(); } + }); + + it('explicit enforceLimits() returns eviction count', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir, { maxFiles: 100, maxBytes: 1024 }); + const a = store.put('a'); ageBlob(store.pathFor(a), 3000); + const b = store.put('b'); ageBlob(store.pathFor(b), 2000); + const c = store.put('c'); ageBlob(store.pathFor(c), 1000); + + // Tighten the cap by reopening the store at a smaller limit. + const tight = openBlobStore(dir, { maxFiles: 1 }); + const evicted = tight.enforceLimits(); + expect(evicted).toBe(2); + expect(tight.stats().count).toBe(1); + expect(tight.exists(c)).toBe(true); + } finally { cleanup(); } + }); +}); + +describe('blob-store: stats', () => { + it('reports count + bytes + caps', () => { + const { dir, cleanup } = workDir(); + try { + const store = openBlobStore(dir, { maxFiles: 50, maxBytes: 999 }); + store.put('xxxx'); + store.put('yyyyy'); + const s = store.stats(); + expect(s.count).toBe(2); + expect(s.bytes).toBe(9); + expect(s.maxFiles).toBe(50); + expect(s.maxBytes).toBe(999); + } finally { cleanup(); } + }); +}); + +// ── helpers ────────────────────────────────────────────────────────────────── + +function ageBlob(path, msAgo) { + const t = new Date(Date.now() - msAgo); + utimesSync(path, t, statSync(path).mtime); +} diff --git a/tests/unit/cac-config.test.js b/tests/unit/cac-config.test.js new file mode 100644 index 0000000..f9c750f --- /dev/null +++ b/tests/unit/cac-config.test.js @@ -0,0 +1,121 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdirSync, rmSync, writeFileSync, readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { + loadCacConfig, saveCacConfig, updateCacConfig, + defaultCacConfig, VALID_MODES, VALID_ACTIONS, +} from '../../src/core/cac-config.js'; + +let projectDir; + +beforeEach(() => { + projectDir = join(tmpdir(), `pos-cac-cfg-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}`); + mkdirSync(projectDir, { recursive: true }); +}); + +afterEach(() => { + try { rmSync(projectDir, { recursive: true, force: true }); } catch {} +}); + +describe('cac-config: defaults + load', () => { + test('default state is disabled, shadow mode', () => { + const def = defaultCacConfig(); + expect(def.enabled).toBe(false); + expect(def.mode).toBe('shadow'); + expect(def.action).toBe('downgrade'); + expect(def.threshold).toBeGreaterThan(0); + expect(def.threshold).toBeLessThan(1); + expect(def.min_samples).toBeGreaterThanOrEqual(1); + }); + + test('VALID_MODES and VALID_ACTIONS exposed for UI', () => { + expect(VALID_MODES).toContain('shadow'); + expect(VALID_MODES).toContain('active'); + expect(VALID_ACTIONS).toContain('downgrade'); + expect(VALID_ACTIONS).toContain('suppress'); + }); + + test('loadCacConfig on missing file returns defaults (no throw)', () => { + const s = loadCacConfig(projectDir); + expect(s).toEqual(defaultCacConfig()); + }); + + test('loadCacConfig on malformed JSON returns defaults + logs', () => { + const path = join(projectDir, '.pos-supervisor', 'cac-config.json'); + mkdirSync(join(projectDir, '.pos-supervisor'), { recursive: true }); + writeFileSync(path, '{not json'); + let logged = null; + const s = loadCacConfig(projectDir, { log: (m) => { logged = m; } }); + expect(s).toEqual(defaultCacConfig()); + expect(logged).toContain('failed to parse'); + }); +}); + +describe('cac-config: save + round-trip', () => { + test('saveCacConfig persists and round-trips', () => { + saveCacConfig(projectDir, { enabled: true, mode: 'active', threshold: 0.5, action: 'suppress', min_samples: 10 }); + const loaded = loadCacConfig(projectDir); + expect(loaded.enabled).toBe(true); + expect(loaded.mode).toBe('active'); + expect(loaded.threshold).toBe(0.5); + expect(loaded.action).toBe('suppress'); + expect(loaded.min_samples).toBe(10); + }); + + test('saveCacConfig coerces invalid mode to default', () => { + saveCacConfig(projectDir, { enabled: true, mode: 'turbo', threshold: 0.4 }); + const loaded = loadCacConfig(projectDir); + expect(loaded.mode).toBe('shadow'); // coerced from invalid + expect(loaded.threshold).toBe(0.4); // valid, kept + expect(loaded.enabled).toBe(true); // valid, kept + }); + + test('saveCacConfig clamps out-of-range threshold to default', () => { + saveCacConfig(projectDir, { threshold: 1.5 }); + const loaded = loadCacConfig(projectDir); + expect(loaded.threshold).toBe(defaultCacConfig().threshold); + }); + + test('saveCacConfig rejects negative min_samples', () => { + saveCacConfig(projectDir, { min_samples: -3 }); + const loaded = loadCacConfig(projectDir); + expect(loaded.min_samples).toBe(defaultCacConfig().min_samples); + }); + + test('saveCacConfig writes valid JSON file with version', () => { + saveCacConfig(projectDir, { enabled: true }); + const path = join(projectDir, '.pos-supervisor', 'cac-config.json'); + const raw = readFileSync(path, 'utf-8'); + const parsed = JSON.parse(raw); + expect(parsed.version).toBe(1); + expect(parsed.enabled).toBe(true); + }); +}); + +describe('cac-config: update (patch)', () => { + test('updateCacConfig merges into existing state', () => { + saveCacConfig(projectDir, { enabled: true, threshold: 0.4 }); + const next = updateCacConfig(projectDir, { mode: 'active' }); + expect(next.enabled).toBe(true); // preserved + expect(next.threshold).toBe(0.4); // preserved + expect(next.mode).toBe('active'); // patched + }); + + test('updateCacConfig drops unknown keys silently', () => { + const next = updateCacConfig(projectDir, { enabled: true, sneaky: 42 }); + expect(next.enabled).toBe(true); + expect(next).not.toHaveProperty('sneaky'); + }); +}); + +describe('cac-config: file-state guarantees', () => { + test('force_enable not object would corrupt rule-overrides — verify cac is robust to similar', () => { + const path = join(projectDir, '.pos-supervisor', 'cac-config.json'); + mkdirSync(join(projectDir, '.pos-supervisor'), { recursive: true }); + writeFileSync(path, JSON.stringify({ version: 1, enabled: 'lol' })); + const loaded = loadCacConfig(projectDir); + // 'lol' is not a boolean → coerced to default + expect(loaded.enabled).toBe(false); + }); +}); diff --git a/tests/unit/cac-predictor.test.js b/tests/unit/cac-predictor.test.js new file mode 100644 index 0000000..5fb50b8 --- /dev/null +++ b/tests/unit/cac-predictor.test.js @@ -0,0 +1,563 @@ +import { describe, test, expect, beforeEach } from 'bun:test'; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { + scoreFixHelpfulness, + decideAction, + applyCac, + getRecentCacDecisions, + clearRecentCacDecisions, + loadRecentCacDecisions, + rehydrateRecentCacDecisions, +} from '../../src/core/cac-predictor.js'; +import { defaultCacConfig } from '../../src/core/cac-config.js'; +import { makeEvent } from '../../src/core/session-events.js'; + +beforeEach(() => { + clearRecentCacDecisions(); +}); + +// ── scoreFixHelpfulness ──────────────────────────────────────────────────── + +describe('scoreFixHelpfulness: hierarchy', () => { + test('uses (rule_id, file_domain) when its sample count meets min_samples', () => { + const historyProvider = (ruleId, domain) => { + if (ruleId === 'A.x' && domain === 'pages') return { adopted: 8, total: 10 }; + if (ruleId === 'A.x' && domain === null) return { adopted: 5, total: 50 }; + return { adopted: 0, total: 0 }; + }; + const r = scoreFixHelpfulness({ + rule_id: 'A.x', + severity: 'error', + file_domain: 'pages', + min_samples: 5, + historyProvider, + severityProvider: () => ({ adopted: 0, total: 0 }), + }); + expect(r.feature).toBe('rule_id+domain'); + expect(r.n_samples).toBe(10); + expect(r.p_adopted).toBeGreaterThan(0.5); + }); + + test('falls back to (rule_id) when (rule_id+domain) is under-sampled', () => { + const historyProvider = (ruleId, domain) => { + if (ruleId === 'A.x' && domain === 'pages') return { adopted: 1, total: 2 }; + if (ruleId === 'A.x' && domain === null) return { adopted: 30, total: 100 }; + return { adopted: 0, total: 0 }; + }; + const r = scoreFixHelpfulness({ + rule_id: 'A.x', + severity: 'warning', + file_domain: 'pages', + min_samples: 5, + historyProvider, + severityProvider: () => ({ adopted: 0, total: 0 }), + }); + expect(r.feature).toBe('rule_id'); + expect(r.n_samples).toBe(100); + }); + + test('falls back to severity when both rule levels are under-sampled', () => { + const r = scoreFixHelpfulness({ + rule_id: 'A.x', + severity: 'error', + file_domain: 'pages', + min_samples: 5, + historyProvider: () => ({ adopted: 1, total: 2 }), + severityProvider: (sev) => sev === 'error' ? { adopted: 100, total: 500 } : { adopted: 0, total: 0 }, + }); + expect(r.feature).toBe('severity'); + expect(r.n_samples).toBe(500); + }); + + test('returns prior with feature="prior" when nothing has signal', () => { + const r = scoreFixHelpfulness({ + rule_id: 'A.x', + severity: 'error', + file_domain: 'pages', + min_samples: 5, + historyProvider: () => ({ adopted: 0, total: 0 }), + severityProvider: () => ({ adopted: 0, total: 0 }), + }); + expect(r.feature).toBe('prior'); + expect(r.p_adopted).toBe(0.5); + expect(r.n_samples).toBe(0); + }); + + test('handles missing rule_id without throwing', () => { + const r = scoreFixHelpfulness({ + rule_id: null, + severity: 'error', + file_domain: 'pages', + min_samples: 5, + historyProvider: () => ({ adopted: 0, total: 0 }), + severityProvider: (sev) => ({ adopted: 50, total: 100 }), + }); + expect(r.feature).toBe('severity'); + expect(r.n_samples).toBe(100); + }); + + test('history provider that throws is treated as zero samples', () => { + const r = scoreFixHelpfulness({ + rule_id: 'A.x', + severity: 'error', + file_domain: 'pages', + min_samples: 5, + historyProvider: () => { throw new Error('db down'); }, + severityProvider: () => ({ adopted: 0, total: 0 }), + }); + expect(r.feature).toBe('prior'); + }); +}); + +// ── decideAction ─────────────────────────────────────────────────────────── + +describe('decideAction', () => { + const cfg = (overrides = {}) => ({ ...defaultCacConfig(), ...overrides }); + + test('feature=prior always allows', () => { + const d = decideAction( + { p_adopted: 0.0, feature: 'prior', n_samples: 0 }, + cfg({ threshold: 0.99, action: 'suppress' }), + ); + expect(d.decision).toBe('allow'); + expect(d.reason).toBe('no_signal'); + }); + + test('p_adopted >= threshold → allow', () => { + const d = decideAction( + { p_adopted: 0.7, feature: 'rule_id', n_samples: 30 }, + cfg({ threshold: 0.5 }), + ); + expect(d.decision).toBe('allow'); + expect(d.reason).toBe('above_threshold'); + }); + + test('p_adopted < threshold + action=suppress → suppress', () => { + const d = decideAction( + { p_adopted: 0.1, feature: 'rule_id', n_samples: 30 }, + cfg({ threshold: 0.3, action: 'suppress' }), + ); + expect(d.decision).toBe('suppress'); + }); + + test('p_adopted < threshold + action=downgrade → downgrade', () => { + const d = decideAction( + { p_adopted: 0.1, feature: 'rule_id', n_samples: 30 }, + cfg({ threshold: 0.3, action: 'downgrade' }), + ); + expect(d.decision).toBe('downgrade'); + }); +}); + +// ── applyCac: integration ────────────────────────────────────────────────── + +function makeResult(diags) { + const result = { errors: [], warnings: [], infos: [] }; + for (const d of diags) { + if (d.severity === 'error') result.errors.push(d); + else if (d.severity === 'warning') result.warnings.push(d); + else result.infos.push(d); + } + return result; +} + +describe('applyCac: gating', () => { + test('disabled config → no-op (result unchanged, no decisions)', () => { + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'm' }, + ]); + const decisions = applyCac(result, { + config: { ...defaultCacConfig(), enabled: false }, + historyProvider: () => ({ adopted: 0, total: 100 }), + }); + expect(result.errors).toHaveLength(1); + expect(decisions).toHaveLength(0); + }); + + test('shadow mode: records decision but never modifies result', () => { + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'm' }, + ]); + const decisions = applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'shadow', threshold: 0.5, action: 'suppress', min_samples: 5 }, + historyProvider: (rid, dom) => rid === 'X.bad' ? { adopted: 0, total: 100 } : { adopted: 0, total: 0 }, + severityProvider: () => ({ adopted: 0, total: 0 }), + filePath: 'app/views/pages/index.html.liquid', + }); + expect(result.errors).toHaveLength(1); // not suppressed + expect(decisions).toHaveLength(1); + expect(decisions[0].decision.decision).toBe('suppress'); // would-be decision + const recorded = getRecentCacDecisions(); + expect(recorded).toHaveLength(1); + expect(recorded[0].mode).toBe('shadow'); + }); + + test('active mode + suppress: drops below-threshold diagnostic', () => { + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }, + { severity: 'error', check: 'Y', rule_id: 'Y.good', message: 'b' }, + { severity: 'warning', check: 'Z', rule_id: 'Z.unk', message: 'c' }, + ]); + const decisions = applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'active', threshold: 0.4, action: 'suppress', min_samples: 5 }, + historyProvider: (rid, dom) => { + if (rid === 'X.bad') return { adopted: 1, total: 100 }; // ~0.03 → suppress + if (rid === 'Y.good') return { adopted: 80, total: 100 }; // ~0.79 → allow + return { adopted: 0, total: 0 }; + }, + severityProvider: () => ({ adopted: 0, total: 0 }), // Z.unk falls to prior → allow + filePath: 'app/views/pages/index.html.liquid', + }); + const remainingChecks = [...result.errors, ...result.warnings, ...result.infos].map(d => d.check); + expect(remainingChecks).not.toContain('X'); + expect(remainingChecks).toContain('Y'); + expect(remainingChecks).toContain('Z'); + const xDec = decisions.find(d => d.check === 'X').decision.decision; + expect(xDec).toBe('suppress'); + }); + + test('active mode + downgrade: reduces severity and rebalances buckets', () => { + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }, + ]); + applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'active', threshold: 0.5, action: 'downgrade', min_samples: 5 }, + historyProvider: () => ({ adopted: 1, total: 100 }), + severityProvider: () => ({ adopted: 0, total: 0 }), + filePath: 'app/views/pages/index.html.liquid', + }); + expect(result.errors).toHaveLength(0); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0].cac_downgraded).toBe(true); + expect(result.warnings[0].severity).toBe('warning'); + }); + + test('active mode: error → warning → info on repeated downgrade', () => { + // Simulate two passes (pretend a rule fires twice on consecutive runs). + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }, + ]); + const cfg = { ...defaultCacConfig(), enabled: true, mode: 'active', threshold: 1.0, action: 'downgrade', min_samples: 5 }; + const provider = () => ({ adopted: 0, total: 100 }); + applyCac(result, { config: cfg, historyProvider: provider, severityProvider: () => ({ adopted: 0, total: 0 }), filePath: 'app/views/pages/i.liquid' }); + expect(result.warnings[0].severity).toBe('warning'); + applyCac(result, { config: cfg, historyProvider: provider, severityProvider: () => ({ adopted: 0, total: 0 }), filePath: 'app/views/pages/i.liquid' }); + expect(result.infos[0].severity).toBe('info'); + }); + + test('predictor failure does not throw — diagnostic passes through', () => { + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }, + ]); + let logged = ''; + const decisions = applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'active', threshold: 0.5, action: 'suppress' }, + historyProvider: () => { throw new Error('db down'); }, + severityProvider: () => { throw new Error('db down'); }, + filePath: 'app/views/pages/i.liquid', + log: (m) => { logged = m; }, + }); + // Even though both providers throw, scorer falls back to prior (allow) so + // the diagnostic survives. Predictor-level failure is caught at the + // scoring boundary via safeProvide. + expect(result.errors).toHaveLength(1); + expect(decisions).toHaveLength(1); + }); + + test('uses check fallback when rule_id missing — synthesizes .unmatched', () => { + const result = makeResult([ + { severity: 'error', check: 'OrphanedPartial', message: 'a' /* no rule_id */ }, + ]); + const seen = []; + applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'active', threshold: 0.5, action: 'suppress', min_samples: 5 }, + historyProvider: (rid, dom) => { seen.push([rid, dom]); return { adopted: 50, total: 100 }; }, + severityProvider: () => ({ adopted: 0, total: 0 }), + filePath: 'app/views/partials/foo.liquid', + }); + expect(seen.some(([rid]) => rid === 'OrphanedPartial.unmatched')).toBe(true); + expect(seen.some(([, dom]) => dom === 'partials')).toBe(true); + }); + + test('passes file_domain derived from filePath to the provider', () => { + const result = makeResult([ + { severity: 'warning', check: 'X', rule_id: 'X.y', message: 'm' }, + ]); + const calls = []; + applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'shadow', threshold: 0.5, min_samples: 5 }, + historyProvider: (rid, dom) => { calls.push([rid, dom]); return { adopted: 0, total: 0 }; }, + severityProvider: () => ({ adopted: 0, total: 0 }), + filePath: 'app/lib/queries/blog_posts/search.graphql', + }); + expect(calls).toContainEqual(['X.y', 'queries']); + expect(calls).toContainEqual(['X.y', null]); + }); +}); + +describe('applyCac: telemetry ring buffer', () => { + test('records up to MAX_RECENT_DECISIONS most-recent entries', () => { + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }, + ]); + const cfg = { ...defaultCacConfig(), enabled: true, mode: 'shadow', threshold: 0.5, min_samples: 5 }; + for (let i = 0; i < 250; i++) { + applyCac(result, { + config: cfg, + historyProvider: () => ({ adopted: 1, total: 100 }), + severityProvider: () => ({ adopted: 0, total: 0 }), + filePath: `app/views/pages/p${i}.liquid`, + }); + } + const recorded = getRecentCacDecisions(); + expect(recorded.length).toBeLessThanOrEqual(200); + expect(recorded.at(-1).file).toContain('p249'); + }); + + test('emits cac_decision events to sessionBus when provided', () => { + const events = []; + const sessionBus = { emit: (kind, payload, ts) => events.push({ kind, payload, ts }) }; + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }, + ]); + applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'shadow', threshold: 0.5, min_samples: 5 }, + historyProvider: () => ({ adopted: 1, total: 100 }), + severityProvider: () => ({ adopted: 0, total: 0 }), + sessionBus, + filePath: 'app/views/pages/p.liquid', + }); + expect(events).toHaveLength(1); + expect(events[0].kind).toBe('cac_decision'); + expect(events[0].payload.decision).toBe('downgrade'); + // Regression: the payload MUST NOT carry a `ts` field — that key is + // reserved by the session-bus envelope (ENVELOPE_KEYS in + // session-events.js). When it slipped through, makeEvent threw and the + // try/catch in recordDecision dropped the event silently. The bus arg + // is the timestamp's only home. + expect(events[0].payload.ts).toBeUndefined(); + expect(typeof events[0].ts).toBe('string'); + expect(events[0].ts).toMatch(/^\d{4}-\d{2}-\d{2}T/); + }); + + test('emit failure does not break the in-memory ring', () => { + // The session bus may throw on its own (writer closed, fsync error, + // misconfigured kind). The predictor's audit trail in memory is the + // dashboard's primary source within a session — it must survive. + const sessionBus = { + emit: () => { throw new Error('writer closed'); }, + }; + const result = makeResult([ + { severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }, + ]); + expect(() => applyCac(result, { + config: { ...defaultCacConfig(), enabled: true, mode: 'shadow', threshold: 0.5, min_samples: 5 }, + historyProvider: () => ({ adopted: 1, total: 100 }), + severityProvider: () => ({ adopted: 0, total: 0 }), + sessionBus, + filePath: 'app/views/pages/p.liquid', + })).not.toThrow(); + expect(getRecentCacDecisions()).toHaveLength(1); + }); +}); + +// ── loadRecentCacDecisions / rehydrateRecentCacDecisions ───────────────────── + +const SID = 'session-2026-04-29T00-00-00-000Z'; + +function writeSessionLog(dir, sessionName, events) { + const sessionDir = join(dir, sessionName); + mkdirSync(sessionDir, { recursive: true }); + const lines = events.map(e => JSON.stringify(e)).join('\n') + '\n'; + writeFileSync(join(sessionDir, 'events.ndjson'), lines, 'utf8'); +} + +function cacDecisionEvent({ session = SID, ts, ...payload }) { + return makeEvent({ + session_id: session, + ts, + kind: 'cac_decision', + payload: { + file: 'app/views/pages/p.liquid', + rule_id: 'X.y', + check: 'X', + severity: 'warning', + file_domain: 'pages', + p_adopted: 0.18, + p_lower: 0.05, + p_upper: 0.45, + n_samples: 7, + feature: 'rule_id', + decision: 'downgrade', + reason: 'below_threshold', + mode: 'shadow', + ...payload, + }, + }); +} + +describe('loadRecentCacDecisions', () => { + let dir; + beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), 'cac-load-')); + }); + + test('returns [] when sessions dir is missing', () => { + expect(loadRecentCacDecisions(join(dir, 'absent'))).toEqual([]); + }); + + test('returns [] when sessions dir is empty', () => { + mkdirSync(join(dir, 'sessions')); + expect(loadRecentCacDecisions(join(dir, 'sessions'))).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }); + + test('reads cac_decision events from one session and returns ring-shape entries', () => { + const events = [ + cacDecisionEvent({ ts: '2026-04-29T01:00:00.000Z', file: 'a.liquid', rule_id: 'A.x' }), + cacDecisionEvent({ ts: '2026-04-29T01:00:01.000Z', file: 'b.liquid', rule_id: 'B.y' }), + ]; + writeSessionLog(dir, SID, events); + + const out = loadRecentCacDecisions(dir); + expect(out).toHaveLength(2); + expect(out[0].ts).toBe('2026-04-29T01:00:00.000Z'); + expect(out[0].file).toBe('a.liquid'); + expect(out[0].rule_id).toBe('A.x'); + // Ring shape: ts is on the entry, payload fields are flattened + expect(out[1].decision).toBe('downgrade'); + expect(out[1].feature).toBe('rule_id'); + rmSync(dir, { recursive: true, force: true }); + }); + + test('skips non-cac_decision lines without crashing', () => { + const mixed = [ + makeEvent({ session_id: SID, ts: '2026-04-29T01:00:00.000Z', kind: 'server_start', + payload: { project_dir: '/x', version: '0.0.0', started_at: '2026-04-29T01:00:00.000Z' } }), + cacDecisionEvent({ ts: '2026-04-29T01:00:01.000Z' }), + makeEvent({ session_id: SID, ts: '2026-04-29T01:00:02.000Z', kind: 'log', + payload: { level: 'info', message: 'hi' } }), + ]; + writeSessionLog(dir, SID, mixed); + + const out = loadRecentCacDecisions(dir); + expect(out).toHaveLength(1); + expect(out[0].ts).toBe('2026-04-29T01:00:01.000Z'); + rmSync(dir, { recursive: true, force: true }); + }); + + test('tolerates malformed JSON lines and partial events', () => { + const valid = cacDecisionEvent({ ts: '2026-04-29T01:00:00.000Z' }); + const sessionDir = join(dir, SID); + mkdirSync(sessionDir, { recursive: true }); + const content = [ + '{not-json', + JSON.stringify(valid), + '', + '{"v":1,"session_id":"x","ts":"2026-04-29T01:00:00.000Z","kind":"cac_decision"}', // missing payload fields + '{"v":99,"kind":"cac_decision"}', // unsupported version + ].join('\n'); + writeSessionLog(dir, SID, []); // ensure dir exists; we then overwrite the file + writeFileSync(join(sessionDir, 'events.ndjson'), content, 'utf8'); + + const out = loadRecentCacDecisions(dir); + expect(out).toHaveLength(1); + expect(out[0].ts).toBe('2026-04-29T01:00:00.000Z'); + rmSync(dir, { recursive: true, force: true }); + }); + + test('merges decisions across multiple sessions in chronological order', () => { + writeSessionLog(dir, 'session-2026-04-28T00-00-00-000Z', [ + cacDecisionEvent({ session: 'session-2026-04-28T00-00-00-000Z', + ts: '2026-04-28T01:00:00.000Z', file: 'old.liquid' }), + ]); + writeSessionLog(dir, 'session-2026-04-29T00-00-00-000Z', [ + cacDecisionEvent({ session: 'session-2026-04-29T00-00-00-000Z', + ts: '2026-04-29T01:00:00.000Z', file: 'new.liquid' }), + ]); + + const out = loadRecentCacDecisions(dir); + expect(out).toHaveLength(2); + expect(out[0].file).toBe('old.liquid'); + expect(out[1].file).toBe('new.liquid'); + rmSync(dir, { recursive: true, force: true }); + }); + + test('respects the limit, keeping the most recent entries', () => { + const events = []; + for (let i = 0; i < 50; i++) { + const tsMs = Date.UTC(2026, 3, 29, 1, 0, i, 0); // sequential second granularity + events.push(cacDecisionEvent({ + ts: new Date(tsMs).toISOString(), + file: `f${i}.liquid`, + })); + } + writeSessionLog(dir, SID, events); + + const out = loadRecentCacDecisions(dir, 10); + expect(out).toHaveLength(10); + // Most recent kept, oldest dropped + expect(out[0].file).toBe('f40.liquid'); + expect(out[9].file).toBe('f49.liquid'); + rmSync(dir, { recursive: true, force: true }); + }); + + test('limit <= 0 returns empty without I/O', () => { + expect(loadRecentCacDecisions(dir, 0)).toEqual([]); + expect(loadRecentCacDecisions(dir, -5)).toEqual([]); + }); +}); + +describe('rehydrateRecentCacDecisions', () => { + let dir; + beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), 'cac-rehydrate-')); + clearRecentCacDecisions(); + }); + + test('replaces the in-memory ring with decisions from disk', () => { + writeSessionLog(dir, SID, [ + cacDecisionEvent({ ts: '2026-04-29T01:00:00.000Z', file: 'a.liquid' }), + cacDecisionEvent({ ts: '2026-04-29T01:00:01.000Z', file: 'b.liquid' }), + ]); + + const n = rehydrateRecentCacDecisions(dir); + expect(n).toBe(2); + const ring = getRecentCacDecisions(); + expect(ring).toHaveLength(2); + expect(ring[0].file).toBe('a.liquid'); + expect(ring[1].file).toBe('b.liquid'); + rmSync(dir, { recursive: true, force: true }); + }); + + test('is idempotent — repeated calls produce the same ring', () => { + writeSessionLog(dir, SID, [ + cacDecisionEvent({ ts: '2026-04-29T01:00:00.000Z' }), + ]); + rehydrateRecentCacDecisions(dir); + rehydrateRecentCacDecisions(dir); + expect(getRecentCacDecisions()).toHaveLength(1); + rmSync(dir, { recursive: true, force: true }); + }); + + test('clears the ring when the sessions dir is empty (no carry-over)', () => { + // Pre-seed via a live emit so the ring has content + applyCac(makeResult([{ severity: 'error', check: 'X', rule_id: 'X.bad', message: 'a' }]), { + config: { ...defaultCacConfig(), enabled: true, mode: 'shadow', threshold: 0.5, min_samples: 5 }, + historyProvider: () => ({ adopted: 1, total: 100 }), + severityProvider: () => ({ adopted: 0, total: 0 }), + filePath: 'app/views/pages/p.liquid', + }); + expect(getRecentCacDecisions().length).toBeGreaterThan(0); + + rehydrateRecentCacDecisions(dir); + expect(getRecentCacDecisions()).toHaveLength(0); + }); + + test('handles missing sessions dir without throwing', () => { + expect(() => rehydrateRecentCacDecisions(join(dir, 'never-existed'))).not.toThrow(); + expect(getRecentCacDecisions()).toHaveLength(0); + }); +}); diff --git a/tests/unit/case-base-integration.test.js b/tests/unit/case-base-integration.test.js new file mode 100644 index 0000000..561df42 --- /dev/null +++ b/tests/unit/case-base-integration.test.js @@ -0,0 +1,159 @@ +import { describe, it, expect, beforeEach, afterEach } from 'bun:test'; +import { registerRules, clearRules, runRules } from '../../src/core/rules/engine.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; +import { setEngineMode, resetEngineMode } from '../../src/core/engine-mode.js'; + +function buildMinimalGraph() { + return buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, + graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); +} + +const testRule = { + id: 'TestCheck.basic', + check: 'TestCheck', + priority: 10, + when: () => true, + apply: () => ({ + rule_id: 'TestCheck.basic', + hint_md: 'test hint', + fixes: [], + confidence: 0.7, + }), +}; + +describe('Phase H: Case-base scoring in rule engine', () => { + beforeEach(() => { + resetEngineMode(); + setEngineMode('adaptive'); + clearRules(); + registerRules([testRule]); + }); + afterEach(() => { resetEngineMode(); }); + + it('runs without analytics store (no scoring applied)', () => { + const graph = buildMinimalGraph(); + const diag = { check: 'TestCheck', params: {}, message: 'test', file: 'test.liquid', line: 1 }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.confidence).toBe(0.7); + expect(result.case_base_signal).toBeUndefined(); + }); + + it('runs with analytics store but no template_fp (no scoring)', () => { + const graph = buildMinimalGraph(); + const mockStore = {}; + const diag = { check: 'TestCheck', params: {}, message: 'test', file: 'test.liquid', line: 1 }; + const result = runRules(diag, { graph, analyticsStore: mockStore }); + expect(result).not.toBeNull(); + expect(result.confidence).toBe(0.7); + expect(result.case_base_signal).toBeUndefined(); + }); + + it('applies positive adjustment from case-base scoring', () => { + const graph = buildMinimalGraph(); + const mockStore = { + queryOne: (sql, params) => { + if (sql.includes('emitted')) return { emitted: 10 }; + return null; + }, + query: (sql, params) => { + return [ + { outcome: 'resolved', cnt: 8 }, + { outcome: 'unchanged', cnt: 2 }, + ]; + }, + }; + const diag = { + check: 'TestCheck', params: {}, message: 'test', + file: 'test.liquid', line: 1, template_fp: 'abc123', + }; + const result = runRules(diag, { graph, analyticsStore: mockStore }); + expect(result).not.toBeNull(); + expect(result.confidence).toBeCloseTo(0.8, 10); + expect(result.case_base_signal).toBeDefined(); + expect(result.case_base_signal.adjustment).toBe(0.1); + expect(result.case_base_signal.reason).toContain('resolution rate'); + }); + + it('applies negative adjustment for high regression rate', () => { + const graph = buildMinimalGraph(); + const mockStore = { + queryOne: (sql, params) => { + if (sql.includes('emitted')) return { emitted: 10 }; + return null; + }, + query: (sql, params) => { + return [ + { outcome: 'resolved', cnt: 2 }, + { outcome: 'regressed', cnt: 5 }, + { outcome: 'unchanged', cnt: 3 }, + ]; + }, + }; + const diag = { + check: 'TestCheck', params: {}, message: 'test', + file: 'test.liquid', line: 1, template_fp: 'abc123', + }; + const result = runRules(diag, { graph, analyticsStore: mockStore }); + expect(result).not.toBeNull(); + expect(result.confidence).toBeCloseTo(0.5, 10); + expect(result.case_base_signal).toBeDefined(); + expect(result.case_base_signal.adjustment).toBe(-0.2); + expect(result.case_base_signal.reason).toContain('regression rate'); + }); + + it('clamps confidence to [0, 1] range', () => { + clearRules(); + registerRules([{ + ...testRule, + apply: () => ({ rule_id: 'TestCheck.basic', hint_md: 'test', fixes: [], confidence: 0.95 }), + }]); + + const graph = buildMinimalGraph(); + const mockStore = { + queryOne: () => ({ emitted: 10 }), + query: () => [{ outcome: 'resolved', cnt: 8 }, { outcome: 'unchanged', cnt: 2 }], + }; + const diag = { + check: 'TestCheck', params: {}, message: 'test', + file: 'test.liquid', line: 1, template_fp: 'abc123', + }; + const result = runRules(diag, { graph, analyticsStore: mockStore }); + expect(result.confidence).toBeLessThanOrEqual(1); + expect(result.confidence).toBe(1); // 0.95 + 0.1 clamped + }); + + it('survives analytics store errors gracefully', () => { + const graph = buildMinimalGraph(); + const brokenStore = { + queryOne: () => { throw new Error('db locked'); }, + query: () => { throw new Error('db locked'); }, + }; + const diag = { + check: 'TestCheck', params: {}, message: 'test', + file: 'test.liquid', line: 1, template_fp: 'abc123', + }; + const result = runRules(diag, { graph, analyticsStore: brokenStore }); + expect(result).not.toBeNull(); + expect(result.confidence).toBe(0.7); // original, no adjustment + expect(result.case_base_signal).toBeUndefined(); + }); + + it('applies scoring in multiMatch mode', () => { + const graph = buildMinimalGraph(); + const mockStore = { + queryOne: () => ({ emitted: 10 }), + query: () => [{ outcome: 'resolved', cnt: 8 }, { outcome: 'unchanged', cnt: 2 }], + }; + const diag = { + check: 'TestCheck', params: {}, message: 'test', + file: 'test.liquid', line: 1, template_fp: 'abc123', + }; + const results = runRules(diag, { graph, analyticsStore: mockStore }, { multiMatch: true }); + expect(results).toHaveLength(1); + expect(results[0].confidence).toBeCloseTo(0.8, 10); + expect(results[0].case_base_signal).toBeDefined(); + }); +}); diff --git a/tests/unit/case-base.test.js b/tests/unit/case-base.test.js new file mode 100644 index 0000000..cfd8ed4 --- /dev/null +++ b/tests/unit/case-base.test.js @@ -0,0 +1,552 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { openAnalyticsStore } from '../../src/core/analytics-store.js'; +import { retrieveCases, retrieveCasesByCheck, ruleScores, scoreRule, suggestedRules, generateRuleTemplate, synthesizeGuardPredicate } from '../../src/core/case-base.js'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +function tmpPath() { + return join(tmpdir(), `pos-case-base-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); +} + +function seedStore(store, diagnostics, outcomes) { + for (const d of diagnostics) { + store.db.prepare(` + INSERT INTO diagnostics (fp, template_fp, session_id, file, check_name, severity, ts, hint_rule_id, content_hash, suppressed) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + `).run(d.fp, d.template_fp ?? null, d.session_id ?? 'sess-1', d.file ?? 'test.liquid', + d.check_name, d.severity ?? 'error', d.ts ?? '2026-04-17T10:00:00Z', + d.hint_rule_id ?? null, d.content_hash ?? null, d.suppressed ?? 0); + } + + for (const w of (outcomes.windows || [])) { + store.db.prepare(` + INSERT INTO windows (id, session_id, file, idx, ts_start, ts_end) + VALUES (?, ?, ?, ?, ?, ?) + `).run(w.id, w.session_id ?? 'sess-1', w.file ?? 'test.liquid', w.idx ?? 0, + w.ts_start ?? '2026-04-17T10:00:00Z', w.ts_end ?? '2026-04-17T10:01:00Z'); + } + + for (const o of (outcomes.outcomes || [])) { + const wid = o.window_id ?? 1; + const w = (outcomes.windows || []).find(w => w.id === wid); + const session_id = o.session_id ?? w?.session_id ?? 'sess-1'; + const file = o.file ?? w?.file ?? 'test.liquid'; + store.db.prepare(` + INSERT OR REPLACE INTO outcomes (fp, window_id, outcome, fix_applied, collateral_added, session_id, file) + VALUES (?, ?, ?, ?, ?, ?, ?) + `).run(o.fp, wid, o.outcome, o.fix_applied ?? null, o.collateral_added ?? 0, session_id, file); + } +} + +describe('Case base — F1: retrieveCases', () => { + let store, dbPath; + + beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); + }); + afterEach(() => { store.close(); }); + + test('returns empty when no diagnostics match', () => { + const result = retrieveCases(store, 'UnknownFilter', 'nonexistent'); + expect(result.total).toBe(0); + expect(result.cases).toEqual([]); + }); + + test('aggregates outcomes by fix_applied', () => { + seedStore(store, + [ + { fp: 'fp1', template_fp: 'tpl1', check_name: 'UnknownFilter' }, + { fp: 'fp2', template_fp: 'tpl1', check_name: 'UnknownFilter' }, + { fp: 'fp3', template_fp: 'tpl1', check_name: 'UnknownFilter' }, + { fp: 'fp4', template_fp: 'tpl1', check_name: 'UnknownFilter' }, + ], + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'fp1', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp2', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp3', outcome: 'regressed', fix_applied: 'verbatim' }, + { fp: 'fp4', outcome: 'unchanged', fix_applied: null }, + ], + } + ); + + const result = retrieveCases(store, 'UnknownFilter', 'tpl1', { minCases: 1 }); + expect(result.total).toBe(4); + expect(result.cases.length).toBe(2); + + const verbatim = result.cases.find(c => c.fix_applied === 'verbatim'); + expect(verbatim.resolved).toBe(2); + expect(verbatim.regressed).toBe(1); + expect(verbatim.resolution_rate).toBeCloseTo(2 / 3, 2); + }); + + test('filters by minCases', () => { + seedStore(store, + [ + { fp: 'fp1', template_fp: 'tpl1', check_name: 'UnknownFilter' }, + { fp: 'fp2', template_fp: 'tpl1', check_name: 'UnknownFilter' }, + ], + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'fp1', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp2', outcome: 'resolved', fix_applied: 'partial' }, + ], + } + ); + + const result = retrieveCases(store, 'UnknownFilter', 'tpl1', { minCases: 3 }); + expect(result.cases.length).toBe(0); + }); + + test('sorts by resolution rate descending', () => { + seedStore(store, + Array.from({ length: 6 }, (_, i) => ({ fp: `fp${i}`, template_fp: 'tpl1', check_name: 'Check1' })), + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'fp0', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp1', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp2', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp3', outcome: 'resolved', fix_applied: 'partial' }, + { fp: 'fp4', outcome: 'regressed', fix_applied: 'partial' }, + { fp: 'fp5', outcome: 'regressed', fix_applied: 'partial' }, + ], + } + ); + + const result = retrieveCases(store, 'Check1', 'tpl1', { minCases: 1 }); + expect(result.cases[0].fix_applied).toBe('verbatim'); + expect(result.cases[0].resolution_rate).toBe(1); + expect(result.cases[1].fix_applied).toBe('partial'); + expect(result.cases[1].resolution_rate).toBeCloseTo(1 / 3, 2); + }); +}); + +describe('Case base — F1: retrieveCasesByCheck', () => { + let store, dbPath; + + beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); + }); + afterEach(() => { store.close(); }); + + test('returns cases grouped by template_fp', () => { + seedStore(store, + [ + { fp: 'fp1', template_fp: 'tplA', check_name: 'Check1' }, + { fp: 'fp2', template_fp: 'tplA', check_name: 'Check1' }, + { fp: 'fp3', template_fp: 'tplA', check_name: 'Check1' }, + { fp: 'fp4', template_fp: 'tplB', check_name: 'Check1' }, + { fp: 'fp5', template_fp: 'tplB', check_name: 'Check1' }, + { fp: 'fp6', template_fp: 'tplB', check_name: 'Check1' }, + ], + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'fp1', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp2', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp3', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp4', outcome: 'unchanged' }, + { fp: 'fp5', outcome: 'unchanged' }, + { fp: 'fp6', outcome: 'unchanged' }, + ], + } + ); + + const results = retrieveCasesByCheck(store, 'Check1', { minCases: 1 }); + expect(results.length).toBe(2); + expect(results[0].template_fp).toBeDefined(); + expect(results[1].template_fp).toBeDefined(); + }); +}); + +describe('Case base — F2: ruleScores', () => { + let store, dbPath; + + beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); + }); + afterEach(() => { store.close(); }); + + test('computes per-rule stats', () => { + seedStore(store, + [ + { fp: 'fp1', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.shopify_filter' }, + { fp: 'fp2', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.shopify_filter' }, + { fp: 'fp3', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.shopify_filter' }, + { fp: 'fp4', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.shopify_filter' }, + { fp: 'fp5', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.shopify_filter' }, + ], + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'fp1', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp2', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'fp3', outcome: 'resolved' }, + { fp: 'fp4', outcome: 'regressed' }, + { fp: 'fp5', outcome: 'unchanged' }, + ], + } + ); + + const scores = ruleScores(store, { minEmitted: 1 }); + expect(scores.length).toBe(1); + + const s = scores[0]; + expect(s.rule_id).toBe('UnknownFilter.shopify_filter'); + expect(s.emitted).toBe(5); + expect(s.resolved).toBe(3); + expect(s.regressed).toBe(1); + expect(s.adopted).toBe(2); + expect(s.resolution_rate).toBeCloseTo(0.6, 2); + expect(s.regression_rate).toBeCloseTo(0.2, 2); + expect(s.effectiveness).toBeCloseTo(0.4, 2); + expect(s.disabled).toBe(false); + }); + + test('marks rules below threshold as disabled', () => { + const diags = []; + const outs = []; + for (let i = 0; i < 10; i++) { + diags.push({ fp: `fp${i}`, template_fp: 'tpl1', check_name: 'BadCheck', hint_rule_id: 'BadCheck.bad_rule' }); + outs.push({ fp: `fp${i}`, outcome: i < 1 ? 'resolved' : 'regressed' }); + } + + seedStore(store, diags, { windows: [{ id: 1 }], outcomes: outs }); + + const scores = ruleScores(store, { minEmitted: 1 }); + expect(scores.length).toBe(1); + expect(scores[0].disabled).toBe(true); + expect(scores[0].effectiveness).toBeLessThan(RULE_DISABLE_THRESHOLD()); + }); + + test('excludes rules below minEmitted', () => { + seedStore(store, + [{ fp: 'fp1', template_fp: 'tpl1', check_name: 'Check1', hint_rule_id: 'Check1.rule1' }], + { windows: [{ id: 1 }], outcomes: [{ fp: 'fp1', outcome: 'resolved' }] } + ); + + const scores = ruleScores(store, { minEmitted: 5 }); + expect(scores.length).toBe(0); + }); + + test('excludes `${check}.unmatched` fallback rule_ids from promotion decisions (A4)', () => { + // Fallback rule_ids set by the diagnostic pipeline don't correspond to a + // registered rule — including them in ruleScores would feed noise into + // syncDisabledRules and probation. Promotion view must stay clean. + seedStore(store, + [ + { fp: 'u1', template_fp: 'tpl1', check_name: 'OrphanCheck', hint_rule_id: 'OrphanCheck.unmatched' }, + { fp: 'u2', template_fp: 'tpl1', check_name: 'OrphanCheck', hint_rule_id: 'OrphanCheck.unmatched' }, + { fp: 'u3', template_fp: 'tpl1', check_name: 'OrphanCheck', hint_rule_id: 'OrphanCheck.unmatched' }, + { fp: 'r1', template_fp: 'tpl1', check_name: 'RealCheck', hint_rule_id: 'RealCheck.real_rule' }, + ], + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'u1', outcome: 'resolved' }, + { fp: 'r1', outcome: 'resolved' }, + ], + } + ); + + const scores = ruleScores(store, { minEmitted: 1 }); + expect(scores.map(s => s.rule_id)).toEqual(['RealCheck.real_rule']); + }); +}); + +describe('Case base — F2: scoreRule', () => { + let store, dbPath; + + beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); + }); + afterEach(() => { store.close(); }); + + test('returns null with insufficient data', () => { + const result = scoreRule(store, 'SomeRule', 'someTpl'); + expect(result).toBeNull(); + }); + + test('returns positive adjustment for high resolution', () => { + const diags = []; + const outs = []; + for (let i = 0; i < 5; i++) { + diags.push({ fp: `fp${i}`, template_fp: 'tpl1', check_name: 'Check1', hint_rule_id: 'Check1.rule1' }); + outs.push({ fp: `fp${i}`, outcome: 'resolved' }); + } + + seedStore(store, diags, { windows: [{ id: 1 }], outcomes: outs }); + + const result = scoreRule(store, 'Check1.rule1', 'tpl1'); + expect(result).not.toBeNull(); + expect(result.adjustment).toBe(0.1); + }); + + test('returns negative adjustment for high regression', () => { + const diags = []; + const outs = []; + for (let i = 0; i < 5; i++) { + diags.push({ fp: `fp${i}`, template_fp: 'tpl1', check_name: 'Check1', hint_rule_id: 'Check1.rule1' }); + outs.push({ fp: `fp${i}`, outcome: i < 2 ? 'resolved' : 'regressed' }); + } + + seedStore(store, diags, { windows: [{ id: 1 }], outcomes: outs }); + + const result = scoreRule(store, 'Check1.rule1', 'tpl1'); + expect(result).not.toBeNull(); + expect(result.adjustment).toBe(-0.2); + }); +}); + +describe('Case base — F3: suggestedRules', () => { + let store, dbPath; + + beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); + }); + afterEach(() => { store.close(); }); + + test('identifies diagnostics without rules but with clear signal', () => { + const diags = []; + const outs = []; + for (let i = 0; i < 6; i++) { + diags.push({ fp: `fp${i}`, template_fp: 'tpl1', check_name: 'NoRuleCheck', hint_rule_id: 'unknown' }); + outs.push({ fp: `fp${i}`, outcome: 'resolved' }); + } + + seedStore(store, diags, { windows: [{ id: 1 }], outcomes: outs }); + + const suggestions = suggestedRules(store, new Set(), { minCases: 3, minResolutionRate: 0.5 }); + expect(suggestions.length).toBe(1); + expect(suggestions[0].check).toBe('NoRuleCheck'); + expect(suggestions[0].resolution_rate).toBe(1); + }); + + test('excludes diagnostics that already have rules', () => { + const diags = []; + const outs = []; + for (let i = 0; i < 6; i++) { + diags.push({ fp: `fp${i}`, template_fp: 'tpl1', check_name: 'RuledCheck', hint_rule_id: 'RuledCheck.rule1' }); + outs.push({ fp: `fp${i}`, outcome: 'resolved' }); + } + + seedStore(store, diags, { windows: [{ id: 1 }], outcomes: outs }); + + const suggestions = suggestedRules(store, new Set(), { minCases: 3 }); + expect(suggestions.length).toBe(0); + }); + + test('excludes low resolution rate', () => { + const diags = []; + const outs = []; + for (let i = 0; i < 6; i++) { + diags.push({ fp: `fp${i}`, template_fp: 'tpl1', check_name: 'LowResCheck', hint_rule_id: 'unknown' }); + outs.push({ fp: `fp${i}`, outcome: i < 1 ? 'resolved' : 'unchanged' }); + } + + seedStore(store, diags, { windows: [{ id: 1 }], outcomes: outs }); + + const suggestions = suggestedRules(store, new Set(), { minCases: 3, minResolutionRate: 0.5 }); + expect(suggestions.length).toBe(0); + }); +}); + +describe('Case base — F3: generateRuleTemplate', () => { + test('produces valid JS code template', () => { + const suggestion = { + check: 'UnknownFilter', + template_fp: 'abcdef1234567890', + resolution_rate: 0.85, + total_outcomes: 20, + sample_file: 'app/views/partials/test.liquid', + }; + + const template = generateRuleTemplate(suggestion); + expect(template).toContain("id: 'UnknownFilter.case_abcdef12'"); + expect(template).toContain("check: 'UnknownFilter'"); + expect(template).toContain('confidence: 0.85'); + expect(template).toContain('85% across 20 outcomes'); + expect(template).toContain('Never auto-merge'); + }); +}); + +function RULE_DISABLE_THRESHOLD() { return 0.15; } + +// ── Reporting baseline (`since`) ───────────────────────────────────────── +// +// case-base reporting paths (retrieveCases*, ruleScores, suggestedRules, +// synthesizeGuardPredicate) accept `opts.since`. Engine paths +// (`scoreRule`, internal `resolveProbation`) MUST NOT — verified by +// observing that scoreRule has no since parameter and engine call sites +// in server.js / server-status.js pass `since: null` explicitly. + +describe('Case base — reporting baseline (`since`)', () => { + let store, dbPath; + + const OLD = '2026-04-01T00:00:00.000Z'; + const NEW = '2026-04-30T00:00:00.000Z'; + const MID = '2026-04-15T00:00:00.000Z'; + + beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); + }); + afterEach(() => { store.close(); }); + + function seedTwoEras() { + // Use seedStore's default session_id 'sess-1' / file 'test.liquid' for + // both diagnostics and the implicit window — case-base joins outcomes + // back to diagnostics on (fp, session_id, file) so they must match. + seedStore(store, + [ + { fp: 'old1', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.typo', ts: OLD }, + { fp: 'old2', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.typo', ts: OLD }, + { fp: 'new1', template_fp: 'tpl1', check_name: 'UnknownFilter', hint_rule_id: 'UnknownFilter.typo', ts: NEW }, + ], + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'old1', outcome: 'regressed' }, + { fp: 'old2', outcome: 'regressed' }, + { fp: 'new1', outcome: 'resolved', fix_applied: 'verbatim' }, + ], + } + ); + } + + test('retrieveCases: ISO since narrows the case set', () => { + seedTwoEras(); + const all = retrieveCases(store, 'UnknownFilter', 'tpl1', { minCases: 1 }); + expect(all.total).toBe(3); + const post = retrieveCases(store, 'UnknownFilter', 'tpl1', { minCases: 1, since: MID }); + expect(post.total).toBe(1); + }); + + test('retrieveCases: meta baseline applies when since omitted', () => { + seedTwoEras(); + store.setBaselineTs(MID); + const post = retrieveCases(store, 'UnknownFilter', 'tpl1', { minCases: 1 }); + expect(post.total).toBe(1); + store.clearBaseline(); + const all = retrieveCases(store, 'UnknownFilter', 'tpl1', { minCases: 1 }); + expect(all.total).toBe(3); + }); + + test('retrieveCases: since=null bypasses meta baseline', () => { + seedTwoEras(); + store.setBaselineTs(MID); + const all = retrieveCases(store, 'UnknownFilter', 'tpl1', { minCases: 1, since: null }); + expect(all.total).toBe(3); + store.clearBaseline(); + }); + + test('retrieveCasesByCheck: forwards since through to retrieveCases', () => { + seedTwoEras(); + const all = retrieveCasesByCheck(store, 'UnknownFilter', { minCases: 1 }); + expect(all[0].total).toBe(3); + const post = retrieveCasesByCheck(store, 'UnknownFilter', { minCases: 1, since: MID }); + expect(post[0].total).toBe(1); + }); + + test('ruleScores: ISO since filters emit + outcome counts', () => { + seedTwoEras(); + const all = ruleScores(store, { minEmitted: 1 }); + expect(all[0].emitted).toBe(3); + expect(all[0].total_outcomes).toBe(3); + + const post = ruleScores(store, { minEmitted: 1, since: MID }); + expect(post[0].emitted).toBe(1); + expect(post[0].total_outcomes).toBe(1); + expect(post[0].resolved).toBe(1); + }); + + test('ruleScores: since=null is the engine-state bypass', () => { + seedTwoEras(); + store.setBaselineTs(MID); + // Default (meta-resolved) sees only post-baseline. + expect(ruleScores(store, { minEmitted: 1 })[0].emitted).toBe(1); + // Explicit bypass sees full history — this is what server.js + + // tools/server-status.js MUST pass for auto-disable / health snapshot. + expect(ruleScores(store, { minEmitted: 1, since: null })[0].emitted).toBe(3); + store.clearBaseline(); + }); + + test('ruleScores: meta baseline does NOT affect engine bypass call', () => { + // Belt-and-braces: even with a baseline set, since:null returns full data. + seedTwoEras(); + store.setBaselineTs(NEW); // narrowest possible — would hide everything + const out = ruleScores(store, { minEmitted: 1, since: null }); + expect(out[0].emitted).toBe(3); + store.clearBaseline(); + }); + + test('suggestedRules: ISO since narrows candidate templates', () => { + // Seed a template whose post-baseline emits don't reach minCases — should + // disappear from suggestions when since=MID. + seedStore(store, + [ + { fp: 'old-1', template_fp: 'tpl-old', check_name: 'OldOnlyCheck', hint_rule_id: 'unknown', ts: OLD }, + { fp: 'old-2', template_fp: 'tpl-old', check_name: 'OldOnlyCheck', hint_rule_id: 'unknown', ts: OLD }, + { fp: 'old-3', template_fp: 'tpl-old', check_name: 'OldOnlyCheck', hint_rule_id: 'unknown', ts: OLD }, + { fp: 'old-4', template_fp: 'tpl-old', check_name: 'OldOnlyCheck', hint_rule_id: 'unknown', ts: OLD }, + { fp: 'old-5', template_fp: 'tpl-old', check_name: 'OldOnlyCheck', hint_rule_id: 'unknown', ts: OLD }, + ], + { + windows: [{ id: 1 }], + outcomes: [ + { fp: 'old-1', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'old-2', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'old-3', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'old-4', outcome: 'resolved', fix_applied: 'verbatim' }, + { fp: 'old-5', outcome: 'resolved', fix_applied: 'verbatim' }, + ], + } + ); + + const all = suggestedRules(store, new Set(), { minCases: 5, minResolutionRate: 0.5 }); + expect(all.find(s => s.check === 'OldOnlyCheck')).toBeDefined(); + + const post = suggestedRules(store, new Set(), { minCases: 5, minResolutionRate: 0.5, since: MID }); + expect(post.find(s => s.check === 'OldOnlyCheck')).toBeUndefined(); + }); + + test('synthesizeGuardPredicate: ISO since narrows the inferred file_type set', () => { + // 5+ pages (would induce file_type=pages), then 0 post-baseline → no guard. + const diags = []; + for (let i = 0; i < 6; i++) { + diags.push({ + fp: `g-${i}`, template_fp: 'tplG', check_name: 'GuardCheck', + file: `app/views/pages/p${i}.liquid`, ts: OLD, hint_rule_id: 'GuardCheck.r', + }); + } + seedStore(store, diags, { windows: [], outcomes: [] }); + + // classifyFileType returns the singular form ('page', not 'pages'). + const all = synthesizeGuardPredicate(store, 'GuardCheck', 'tplG', { minSamples: 5 }); + expect(all.file_type).toBe('page'); + + const post = synthesizeGuardPredicate(store, 'GuardCheck', 'tplG', { minSamples: 5, since: MID }); + expect(post.file_type).toBeUndefined(); + }); + + test('scoreRule: NO since parameter — always sees full history', () => { + // scoreRule's signature deliberately has no since param. Even after the + // operator sets a baseline, scoreRule sees the full case set so live + // confidence-adjustment never deteriorates from a narrow window. + seedTwoEras(); + store.setBaselineTs(NEW); + const adj = scoreRule(store, 'UnknownFilter.typo', 'tpl1'); + expect(adj).not.toBeNull(); + // 3 cases, 1 resolved, 2 regressed → regression rate 0.67 → harmful adjustment + expect(adj.adjustment).toBeLessThan(0); + store.clearBaseline(); + }); +}); diff --git a/tests/unit/dependency-graph.test.js b/tests/unit/dependency-graph.test.js index b248eaa..6f084b3 100644 --- a/tests/unit/dependency-graph.test.js +++ b/tests/unit/dependency-graph.test.js @@ -1,5 +1,5 @@ /** - * dependency-graph unit tests — pins the edge resolution and dead code + * dependency-graph unit tests — pins the edge resolution and orphaned file * classification that Phase 2.2 introduces. These are pure-function tests * against a synthetic project_map shape — no LSP, no filesystem. */ @@ -7,7 +7,7 @@ import { describe, it, expect } from 'bun:test'; import { buildDependencyGraph, - detectDeadCode, + detectOrphanedFiles, resolveRenderTarget, resolveFunctionTarget, resolveGraphqlTarget, @@ -193,9 +193,9 @@ describe('dependency-graph: buildDependencyGraph', () => { }); }); -// ── Dead code ──────────────────────────────────────────────────────────────── +// ── Orphaned files ─────────────────────────────────────────────────────────── -describe('dependency-graph: detectDeadCode', () => { +describe('dependency-graph: detectOrphanedFiles', () => { it('flags a partial that nothing references', () => { const map = { pages: {}, partials: { 'unused': { path: 'app/views/partials/unused.liquid' } }, @@ -204,7 +204,7 @@ describe('dependency-graph: detectDeadCode', () => { const graph = { 'app/views/partials/unused.liquid': { depends_on: [], referenced_by: [] }, }; - expect(detectDeadCode(graph, map)).toContain('app/views/partials/unused.liquid'); + expect(detectOrphanedFiles(graph, map)).toContain('app/views/partials/unused.liquid'); }); it('never flags a page as dead — pages are HTTP entry points', () => { @@ -215,7 +215,7 @@ describe('dependency-graph: detectDeadCode', () => { const graph = { 'app/views/pages/p.html.liquid': { depends_on: [], referenced_by: [] }, }; - expect(detectDeadCode(graph, map)).not.toContain('app/views/pages/p.html.liquid'); + expect(detectOrphanedFiles(graph, map)).not.toContain('app/views/pages/p.html.liquid'); }); it('exempts build/check/execute subphase files anywhere in the path', () => { @@ -225,7 +225,7 @@ describe('dependency-graph: detectDeadCode', () => { 'app/lib/commands/blog_posts/update/check.liquid': { depends_on: [], referenced_by: [] }, 'app/lib/commands/execute.liquid': { depends_on: [], referenced_by: [] }, }; - const dead = detectDeadCode(graph, map); + const dead = detectOrphanedFiles(graph, map); expect(dead).not.toContain('app/lib/commands/blog_posts/create/build.liquid'); expect(dead).not.toContain('app/lib/commands/blog_posts/update/check.liquid'); expect(dead).not.toContain('app/lib/commands/execute.liquid'); @@ -241,7 +241,7 @@ describe('dependency-graph: detectDeadCode', () => { const graph = { 'app/lib/commands/blog_posts/create.liquid': { depends_on: [], referenced_by: [] }, }; - expect(detectDeadCode(graph, map)).toContain('app/lib/commands/blog_posts/create.liquid'); + expect(detectOrphanedFiles(graph, map)).toContain('app/lib/commands/blog_posts/create.liquid'); }); it('does NOT flag a command whose caller is in the graph', () => { @@ -255,7 +255,7 @@ describe('dependency-graph: detectDeadCode', () => { 'app/views/pages/p.html.liquid': { depends_on: ['app/lib/commands/x/create.liquid'], referenced_by: [] }, 'app/lib/commands/x/create.liquid': { depends_on: [], referenced_by: ['app/views/pages/p.html.liquid'] }, }; - expect(detectDeadCode(graph, map)).not.toContain('app/lib/commands/x/create.liquid'); + expect(detectOrphanedFiles(graph, map)).not.toContain('app/lib/commands/x/create.liquid'); }); it('skips external module paths (they are read-only)', () => { @@ -263,6 +263,17 @@ describe('dependency-graph: detectDeadCode', () => { const graph = { 'modules/user/helpers/thing.liquid': { depends_on: [], referenced_by: [] }, }; - expect(detectDeadCode(graph, map)).toEqual([]); + expect(detectOrphanedFiles(graph, map)).toEqual([]); + }); + + it('never flags translation files as orphaned — they have no liquid callers by design', () => { + const map = { pages: {}, partials: {}, commands: {}, queries: {} }; + const graph = { + 'app/translations/en.yml': { depends_on: [], referenced_by: [] }, + 'app/translations/de.yml': { depends_on: [], referenced_by: [] }, + }; + const dead = detectOrphanedFiles(graph, map); + expect(dead).not.toContain('app/translations/en.yml'); + expect(dead).not.toContain('app/translations/de.yml'); }); }); diff --git a/tests/unit/diagnostic-pipeline-frontmatter-dedup.test.js b/tests/unit/diagnostic-pipeline-frontmatter-dedup.test.js new file mode 100644 index 0000000..08b6ae2 --- /dev/null +++ b/tests/unit/diagnostic-pipeline-frontmatter-dedup.test.js @@ -0,0 +1,171 @@ +/** + * Suppression of upstream `ValidFrontmatter` rows that overlap with our + * richer `pos-supervisor:InvalidLayout` / `pos-supervisor:InvalidFrontMatter` + * structural checks (pos-cli 6.0.7 alignment, 2026-04-25). + * + * Line-anchored: YAML frontmatter is one key per line, so a line collision + * is a reliable signal of the same root cause. + */ + +import { describe, it, expect } from 'bun:test'; +import { suppressUpstreamFrontmatterDup } from '../../src/core/diagnostic-pipeline.js'; + +function makeResult({ errors = [], warnings = [], infos = [] } = {}) { + return { errors: [...errors], warnings: [...warnings], infos: [...infos] }; +} + +describe('suppressUpstreamFrontmatterDup', () => { + it('drops ValidFrontmatter when pos-supervisor:InvalidLayout shares its line', () => { + const result = makeResult({ + warnings: [ + { + check: 'ValidFrontmatter', + severity: 'warning', + message: "Layout 'nonexistent_layout_xyz' does not exist", + line: 3, + }, + { + check: 'pos-supervisor:InvalidLayout', + severity: 'warning', + message: 'Layout `nonexistent_layout_xyz` not found. Expected file: …', + line: 3, + }, + ], + }); + + const removed = suppressUpstreamFrontmatterDup(result); + + expect(removed).toBe(1); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0].check).toBe('pos-supervisor:InvalidLayout'); + expect(result.infos.some(i => i.check === 'pos-supervisor:DuplicateFrontmatterCheck')).toBe(true); + }); + + it('drops ValidFrontmatter when pos-supervisor:InvalidFrontMatter shares its line (error severity)', () => { + const result = makeResult({ + errors: [ + { + check: 'pos-supervisor:InvalidFrontMatter', + severity: 'error', + message: '`cache` is not a front matter option. Use `{% cache key, expire: 3600 %}`.', + line: 3, + }, + ], + warnings: [ + { + check: 'ValidFrontmatter', + severity: 'warning', + message: "Unknown frontmatter field 'cache' in Page file", + line: 3, + }, + ], + }); + + const removed = suppressUpstreamFrontmatterDup(result); + + expect(removed).toBe(1); + expect(result.errors).toHaveLength(1); + expect(result.warnings).toHaveLength(0); + }); + + it('keeps ValidFrontmatter rows that do NOT overlap with our checks', () => { + // Upstream catches deprecated `layout_name` — we don't have a structural + // check for this, so the warning should survive untouched. + const result = makeResult({ + warnings: [ + { + check: 'ValidFrontmatter', + severity: 'warning', + message: "Use 'layout' instead of deprecated 'layout_name'", + line: 4, + }, + { + check: 'pos-supervisor:InvalidLayout', + severity: 'warning', + message: 'Layout `application` not found.', + line: 2, + }, + ], + }); + + const removed = suppressUpstreamFrontmatterDup(result); + + expect(removed).toBe(0); + expect(result.warnings).toHaveLength(2); + expect(result.infos).toHaveLength(0); + }); + + it('is a no-op when no pos-supervisor structural check is present', () => { + const result = makeResult({ + warnings: [ + { + check: 'ValidFrontmatter', + severity: 'warning', + message: "Layout 'foo' does not exist", + line: 3, + }, + ], + }); + + const removed = suppressUpstreamFrontmatterDup(result); + + expect(removed).toBe(0); + expect(result.warnings).toHaveLength(1); + expect(result.infos).toHaveLength(0); + }); + + it('is a no-op when no ValidFrontmatter row is present', () => { + const result = makeResult({ + warnings: [ + { + check: 'pos-supervisor:InvalidLayout', + severity: 'warning', + message: 'Layout `application` not found.', + line: 3, + }, + ], + }); + + const removed = suppressUpstreamFrontmatterDup(result); + + expect(removed).toBe(0); + expect(result.warnings).toHaveLength(1); + expect(result.infos).toHaveLength(0); + }); + + it('idempotent — second call after dedup is a no-op', () => { + const result = makeResult({ + warnings: [ + { check: 'ValidFrontmatter', severity: 'warning', message: 'x', line: 3 }, + { check: 'pos-supervisor:InvalidLayout', severity: 'warning', message: 'y', line: 3 }, + ], + }); + + expect(suppressUpstreamFrontmatterDup(result)).toBe(1); + expect(suppressUpstreamFrontmatterDup(result)).toBe(0); + expect(result.warnings).toHaveLength(1); + // Only one info note was added — no duplicate from the second call. + expect(result.infos.filter(i => i.check === 'pos-supervisor:DuplicateFrontmatterCheck')) + .toHaveLength(1); + }); + + it('drops both ValidFrontmatter rows when multiple of our checks fire', () => { + const result = makeResult({ + errors: [ + { check: 'pos-supervisor:InvalidFrontMatter', severity: 'error', message: 'a', line: 3 }, + ], + warnings: [ + { check: 'pos-supervisor:InvalidLayout', severity: 'warning', message: 'b', line: 4 }, + { check: 'ValidFrontmatter', severity: 'warning', message: 'a-upstream', line: 3 }, + { check: 'ValidFrontmatter', severity: 'warning', message: 'b-upstream', line: 4 }, + { check: 'ValidFrontmatter', severity: 'warning', message: 'novel-upstream', line: 5 }, + ], + }); + + const removed = suppressUpstreamFrontmatterDup(result); + + expect(removed).toBe(2); + // The line-5 ValidFrontmatter is novel and survives. + expect(result.warnings.find(w => w.check === 'ValidFrontmatter').line).toBe(5); + }); +}); diff --git a/tests/unit/diagnostic-pipeline.test.js b/tests/unit/diagnostic-pipeline.test.js index 6972560..5ef2d92 100644 --- a/tests/unit/diagnostic-pipeline.test.js +++ b/tests/unit/diagnostic-pipeline.test.js @@ -7,6 +7,7 @@ import { suppressByPending, buildPendingPartialNames, buildPendingPageKeys, + stampDefaultsOn, } from '../../src/core/diagnostic-pipeline.js'; // ── helpers ────────────────────────────────────────────────────────────────── @@ -247,6 +248,79 @@ describe('diagnostic-pipeline: pending suppression via runDiagnosticPipeline', ( }); }); +// ── verifyMissingPartialsOnDisk: lib/-prefix correctness ──────────────────── +// +// Regression: the resolver used to strip a leading `lib/` before the disk +// check, which routed `lib/commands/X` to `app/lib/commands/X.liquid` and +// silently suppressed the LSP's correct MissingPartial when that bare-form +// file existed. Net effect: the agent saw "no problem" while platformOS +// would 500 at runtime because `lib/commands/X` resolves to +// `app/lib/lib/commands/X.liquid` (the partial search paths are +// `app/views/partials/` and `app/lib/`, not project root). The resolver +// now mirrors upstream `DocumentsLocator` exactly — no prefix stripping — +// so the LSP error survives all the way to the agent. + +describe('diagnostic-pipeline: verifyMissingPartialsOnDisk does not strip `lib/` prefix', () => { + let tmpDir; + + beforeAll(() => { + tmpDir = mkdtempSync(join(tmpdir(), 'pipeline-libpref-')); + mkdirSync(join(tmpDir, 'app/lib/commands/contacts'), { recursive: true }); + writeFileSync( + join(tmpDir, 'app/lib/commands/contacts/create.liquid'), + '{% doc %}{% enddoc %}', + 'utf8', + ); + mkdirSync(join(tmpDir, 'app/views/partials/cards'), { recursive: true }); + writeFileSync( + join(tmpDir, 'app/views/partials/cards/product.liquid'), + '
', + 'utf8', + ); + }); + + afterAll(() => { if (tmpDir) rmSync(tmpDir, { recursive: true, force: true }); }); + + it('suppresses MissingPartial for the bare `commands/X` form when X.liquid is on disk (LSP cache lag)', () => { + const result = makeResult([ + { check: 'MissingPartial', severity: 'error', message: "'commands/contacts/create' does not exist" }, + ]); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/contacts/new.html.liquid', content: '', projectDir: tmpDir }); + expect(result.errors).toHaveLength(0); + expect(result.infos.some(i => i.check === 'pos-supervisor:MissingPartialSuppressed')).toBe(true); + }); + + it('does NOT suppress MissingPartial for the `lib/commands/X` form — the `lib/` prefix expands to `app/lib/lib/...`', () => { + const result = makeResult([ + { check: 'MissingPartial', severity: 'error', message: "'lib/commands/contacts/create' does not exist" }, + ]); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/contacts/new.html.liquid', content: '', projectDir: tmpDir }); + expect(result.errors).toHaveLength(1); + expect(result.errors[0].message).toContain('lib/commands/contacts/create'); + expect(result.infos.some(i => i.check === 'pos-supervisor:MissingPartialSuppressed')).toBe(false); + }); + + it('does NOT suppress MissingPartial for the `lib/queries/X` form even when the bare-form file exists on disk', () => { + mkdirSync(join(tmpDir, 'app/lib/queries/products'), { recursive: true }); + writeFileSync(join(tmpDir, 'app/lib/queries/products/find.liquid'), '{% doc %}{% enddoc %}', 'utf8'); + const result = makeResult([ + { check: 'MissingPartial', severity: 'error', message: "'lib/queries/products/find' does not exist" }, + ]); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/products/show.html.liquid', content: '', projectDir: tmpDir }); + expect(result.errors).toHaveLength(1); + expect(result.infos.some(i => i.check === 'pos-supervisor:MissingPartialSuppressed')).toBe(false); + }); + + it('still suppresses real partial cache-lag misses (non-`lib/` paths)', () => { + const result = makeResult([ + { check: 'MissingPartial', severity: 'error', message: "'cards/product' does not exist" }, + ]); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/index.html.liquid', content: '', projectDir: tmpDir }); + expect(result.errors).toHaveLength(0); + expect(result.infos.some(i => i.check === 'pos-supervisor:MissingPartialSuppressed')).toBe(true); + }); +}); + // ── verifyMissingAssets ────────────────────────────────────────────────────── describe('diagnostic-pipeline: verifyMissingAssets via runDiagnosticPipeline', () => { @@ -571,3 +645,361 @@ describe('verifyOrphanedPartialOnDisk via runDiagnosticPipeline', () => { expect(result.warnings).toHaveLength(1); }); }); + +// ── populateDefaultConfidence (A2) ────────────────────────────────────────── + +describe('diagnostic-pipeline: populateDefaultConfidence', () => { + it('stamps severity-based defaults when the rule engine left confidence unset', () => { + const result = makeResult( + [{ check: 'UndefinedObject', severity: 'error', message: 'foo' }], + [{ check: 'UnusedAssign', severity: 'warning', message: 'bar' }], + [{ check: 'InfoOnly', severity: 'info', message: 'baz' }], + ); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.errors[0].confidence).toBe(0.9); + expect(result.warnings[0].confidence).toBe(0.7); + expect(result.infos[0].confidence).toBe(0.5); + }); + + it('does not overwrite a confidence value that the rule engine already set', () => { + const result = makeResult( + [{ check: 'UndefinedObject', severity: 'error', message: 'foo', confidence: 0.42 }], + ); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.errors[0].confidence).toBe(0.42); + }); + + it('stamps structural default for pos-supervisor: prefixed checks', () => { + const result = makeResult( + [], + [{ check: 'pos-supervisor:RemovedRender', severity: 'warning', message: 'removed' }], + ); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.warnings[0].confidence).toBe(0.75); + }); + + it('runs after suppression — items removed from result never gain a default', () => { + const result = makeResult( + [], + [{ check: 'MissingPartial', severity: 'warning', message: "Missing partial 'notes/show'" }], + ); + runDiagnosticPipeline(result, { + filePath: 'app/views/pages/x.liquid', + content: '', + pendingFiles: ['app/views/partials/notes/show.liquid'], + }); + expect(result.warnings).toHaveLength(0); + }); + + it('falls back to warning-level confidence when severity is unset or unknown', () => { + const result = makeResult( + [], + [{ check: 'Weirdo', message: 'no severity' }], + ); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.warnings[0].confidence).toBe(0.7); + }); + + // ── A4: rule_id fallback ─────────────────────────────────────────────── + it('stamps rule_id as `${check}.unmatched` when no rule fired', () => { + const result = makeResult( + [{ check: 'UndefinedObject', severity: 'error', message: 'foo' }], + [{ check: 'UnusedAssign', severity: 'warning', message: 'bar' }], + ); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.errors[0].rule_id).toBe('UndefinedObject.unmatched'); + expect(result.warnings[0].rule_id).toBe('UnusedAssign.unmatched'); + }); + + it('preserves rule_id set by the rule engine', () => { + const result = makeResult( + [{ check: 'UndefinedObject', severity: 'error', message: 'foo', rule_id: 'UndefinedObject.context_user' }], + ); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.errors[0].rule_id).toBe('UndefinedObject.context_user'); + }); + + it('falls back to `unknown.unmatched` when the diagnostic has no check name', () => { + const result = makeResult( + [], + [{ severity: 'warning', message: 'orphan' }], + ); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.warnings[0].rule_id).toBe('unknown.unmatched'); + }); +}); + +// ── stampDefaultsOn: post-pipeline stamping (confidence-bug fix) ───────────── + +describe('stampDefaultsOn: late-push diagnostics get default confidence', () => { + it('stamps diagnostics added AFTER runDiagnosticPipeline has already run', () => { + const result = makeResult([{ check: 'UnknownFilter', severity: 'error', message: 'x' }]); + runDiagnosticPipeline(result, { filePath: 'app/views/pages/x.liquid', content: '' }); + expect(result.errors[0].confidence).toBe(0.9); + + // Simulate a late push — e.g. structural-warnings / schema validator. + result.warnings.push({ + check: 'pos-supervisor:HtmlInPage', + severity: 'warning', + message: 'HTML in page', + }); + // Without the fix the late row would stay at confidence: null. + stampDefaultsOn(result); + expect(result.warnings[0].confidence).toBe(0.75); // structural default + expect(result.warnings[0].rule_id).toBe('pos-supervisor:HtmlInPage.unmatched'); + }); + + it('is idempotent — re-stamping does not overwrite existing values', () => { + const result = makeResult([ + { check: 'UnknownFilter', severity: 'error', message: 'x', confidence: 0.42, rule_id: 'UnknownFilter.typo' }, + ]); + stampDefaultsOn(result); + expect(result.errors[0].confidence).toBe(0.42); + expect(result.errors[0].rule_id).toBe('UnknownFilter.typo'); + }); +}); + +// ── suppressLspKnownFalsePositives ────────────────────────────────────────── +// +// Pins the LSP "Syntax is not supported" suppression on `assign x = a b` +// boolean comparisons. Upstream pos-cli LSP rejects this construct even +// though `pos-cli check run` and the platformOS Liquid parser both accept +// it. Without the suppression, agents are forced to rewrite valid code as a +// multi-line if/else just to clear the must_fix_before_write gate. + +describe('diagnostic-pipeline: suppressLspKnownFalsePositives', () => { + function syntaxErr(line, message = 'Syntax is not supported') { + return { check: 'LiquidHTMLSyntaxError', severity: 'error', line, message }; + } + + it('suppresses the LSP false positive on `assign x = a == b` when the file parses cleanly', () => { + const content = [ + '{% doc %}', + ' @param {object} object', + '{% enddoc %}', + '{% liquid', + ' assign c = object.errors | default: empty', + ' assign object.valid = c == empty', + ' return object', + '%}', + ].join('\n'); + + const result = makeResult([syntaxErr(6)]); + runDiagnosticPipeline(result, { + filePath: 'app/lib/commands/contacts/create/check.liquid', + content, + }); + + expect(result.errors).toHaveLength(0); + const info = result.infos.find(i => i.check === 'pos-supervisor:LspSyntaxFalsePositiveSuppressed'); + expect(info).toBeDefined(); + expect(info.message).toContain('line(s) 6'); + expect(info.message).toContain('@platformos/liquid-html-parser'); + }); + + it('suppresses every "Syntax is not supported" diagnostic in the same file at once', () => { + const content = [ + '{% liquid', + ' assign a = 1 == 1', + ' assign b = 2 != 3', + '%}', + ].join('\n'); + + const result = makeResult([syntaxErr(2), syntaxErr(3)]); + runDiagnosticPipeline(result, { + filePath: 'app/views/partials/check.liquid', + content, + }); + + expect(result.errors).toHaveLength(0); + const info = result.infos.find(i => i.check === 'pos-supervisor:LspSyntaxFalsePositiveSuppressed'); + expect(info.message).toContain('line(s) 2, 3'); + }); + + it('does NOT suppress when the file has a real syntax error elsewhere (parser fails)', () => { + const content = [ + '{% liquid', + ' assign x = 1 == 1', + '%}', + '{% if foo %}', + ' hello', + '{# missing endif — strict parse fails here #}', + ].join('\n'); + + const result = makeResult([syntaxErr(2)]); + runDiagnosticPipeline(result, { + filePath: 'app/views/partials/broken.liquid', + content, + }); + + expect(result.errors).toHaveLength(1); + expect(result.errors[0].check).toBe('LiquidHTMLSyntaxError'); + expect(result.infos.some(i => i.check === 'pos-supervisor:LspSyntaxFalsePositiveSuppressed')).toBe(false); + }); + + it('does NOT suppress LiquidHTMLSyntaxError diagnostics with a different upstream message', () => { + const content = [ + '{% liquid', + ' assign x = 1', + '%}', + ].join('\n'); + + const result = makeResult([ + { check: 'LiquidHTMLSyntaxError', severity: 'error', line: 1, message: "Invalid syntax for tag 'render'" }, + ]); + runDiagnosticPipeline(result, { + filePath: 'app/views/partials/x.liquid', + content, + }); + + expect(result.errors).toHaveLength(1); + expect(result.infos.some(i => i.check === 'pos-supervisor:LspSyntaxFalsePositiveSuppressed')).toBe(false); + }); + + it('does NOT suppress non-LiquidHTMLSyntaxError checks even when the message text matches', () => { + const content = '{% liquid\n assign x = 1\n%}\n'; + + const result = makeResult([ + { check: 'UnknownFilter', severity: 'error', line: 1, message: 'Syntax is not supported' }, + ]); + runDiagnosticPipeline(result, { + filePath: 'app/views/partials/x.liquid', + content, + }); + + expect(result.errors).toHaveLength(1); + expect(result.errors[0].check).toBe('UnknownFilter'); + }); + + it('also handles diagnostics surfaced as warnings, not just errors', () => { + const content = '{% liquid\n assign x = 1 == 1\n%}\n'; + + const result = makeResult([], [ + { check: 'LiquidHTMLSyntaxError', severity: 'warning', line: 2, message: 'Syntax is not supported' }, + ]); + runDiagnosticPipeline(result, { + filePath: 'app/views/partials/x.liquid', + content, + }); + + expect(result.warnings).toHaveLength(0); + expect(result.infos.some(i => i.check === 'pos-supervisor:LspSyntaxFalsePositiveSuppressed')).toBe(true); + }); +}); + +// ── verifyPageRoutesOnDisk: in-memory file overlay ────────────────────────── +// +// Pins the self-page suppression: when an agent runs validate_code on a +// page whose in-memory frontmatter declares the very (slug, method) pair +// the LSP is complaining about, the route index must reflect the +// in-memory version, not the older on-disk one. Without this overlay the +// agent sees a MissingPage warning for a route the file IS about to serve +// the moment it lands on disk — exactly the false positive observed in +// the DEMO project (POST `/` warning while `app/views/pages/index.liquid` +// declared `method: post` in-memory). + +describe('diagnostic-pipeline: verifyPageRoutesOnDisk respects in-memory overlay', () => { + let tmpDir; + + beforeAll(() => { + tmpDir = mkdtempSync(join(tmpdir(), 'pipeline-route-overlay-')); + mkdirSync(join(tmpDir, 'app/views/pages'), { recursive: true }); + // Disk version: GET / only — no method declared. + writeFileSync( + join(tmpDir, 'app/views/pages/index.liquid'), + '

old version (no frontmatter)

\n', + 'utf8', + ); + }); + + afterAll(() => { if (tmpDir) rmSync(tmpDir, { recursive: true, force: true }); }); + + it("suppresses MissingPage for route '/' (POST) when the file under validation declares method: post in-memory", () => { + const inMemory = [ + '---', + 'method: post', + 'metadata:', + ' title: "Home"', + '---', + '

POST handler in-memory

', + ].join('\n'); + + const result = makeResult([], [ + { check: 'MissingPage', severity: 'warning', line: 6, column: 0, message: "No page found for route '/' (POST)" }, + ]); + runDiagnosticPipeline(result, { + filePath: 'app/views/pages/index.liquid', + content: inMemory, + projectDir: tmpDir, + }); + + expect(result.warnings).toHaveLength(0); + expect(result.infos.some(i => i.check === 'pos-supervisor:MissingPageSuppressed')).toBe(true); + }); + + it('still flags MissingPage when the in-memory frontmatter does not cover the reported method', () => { + const inMemory = [ + '---', + 'method: get', + '---', + '

GET only

', + ].join('\n'); + + const result = makeResult([], [ + { check: 'MissingPage', severity: 'warning', line: 4, column: 0, message: "No page found for route '/' (POST)" }, + ]); + runDiagnosticPipeline(result, { + filePath: 'app/views/pages/index.liquid', + content: inMemory, + projectDir: tmpDir, + }); + + expect(result.warnings).toHaveLength(1); + // wrong-method enrichment — the route IS served, just for GET. + expect(result.warnings[0].hint).toContain('GET'); + }); + + it('treats a brand-new page (not yet on disk) as serving its declared route', () => { + const inMemory = [ + '---', + 'slug: contact', + 'method: post', + '---', + '

new page

', + ].join('\n'); + + const result = makeResult([], [ + { check: 'MissingPage', severity: 'warning', line: 5, column: 0, message: "No page found for route '/contact' (POST)" }, + ]); + runDiagnosticPipeline(result, { + filePath: 'app/views/pages/contact.liquid', + content: inMemory, + projectDir: tmpDir, + }); + + expect(result.warnings).toHaveLength(0); + }); + + it('ignores the overlay when the file under validation is not under app/views/pages/ (partial / layout)', () => { + // A partial cannot serve a route. Even if it has frontmatter (it shouldn't), + // the route index must remain disk-only for non-page files. + const inMemory = [ + '---', + 'slug: pretend', + 'method: post', + '---', + '

partial pretending to be a page

', + ].join('\n'); + + const result = makeResult([], [ + { check: 'MissingPage', severity: 'warning', line: 5, column: 0, message: "No page found for route '/pretend' (POST)" }, + ]); + runDiagnosticPipeline(result, { + filePath: 'app/views/partials/pretend.liquid', + content: inMemory, + projectDir: tmpDir, + }); + + expect(result.warnings).toHaveLength(1); + }); +}); diff --git a/tests/unit/diagnostic-record.test.js b/tests/unit/diagnostic-record.test.js new file mode 100644 index 0000000..feae366 --- /dev/null +++ b/tests/unit/diagnostic-record.test.js @@ -0,0 +1,284 @@ +/** + * diagnostic-record unit tests — pin the typed builder, the param extractor + * registry, and the message-template masking algorithm. Fingerprint stability + * across instances is owned by tests/upstream/diagnostic-fingerprint.test.js; + * this file focuses on the per-check extraction contract. + */ + +import { describe, it, expect } from 'bun:test'; +import { + DIAGNOSTIC_RECORD_VERSION, + KNOWN_EXTRACTOR_CHECKS, + makeDiagnosticRecord, + fingerprint, + templateFingerprint, + messageTemplate, + templateOf, + extractParams, +} from '../../src/core/diagnostic-record.js'; + +describe('diagnostic-record: messageTemplate masking', () => { + it('masks single-quoted identifiers', () => { + expect(messageTemplate("Variable 'foo' is undefined")) + .toBe('Variable is undefined'); + }); + + it('masks double-quoted identifiers', () => { + expect(messageTemplate('Cannot find "products/index"')) + .toBe('Cannot find '); + }); + + it('masks backticked identifiers', () => { + expect(messageTemplate('Use `render` instead of `include`')) + .toBe('Use instead of '); + }); + + it('masks bare integers and floats', () => { + expect(messageTemplate('Line 42 column 7.5 broken')) + .toBe('Line column broken'); + }); + + it('masks hex literals', () => { + expect(messageTemplate('Color #fff value 0xff is invalid')) + .toBe('Color #fff value is invalid'); + }); + + it('does not chew embedded numerics inside identifiers', () => { + // "html5" stays intact (not "html") because the regex is word-anchored. + expect(messageTemplate('html5 doctype required')).toBe('html5 doctype required'); + }); + + it('collapses runs of whitespace and trims', () => { + expect(messageTemplate(' foo bar ')).toBe('foo bar'); + }); + + it('returns empty string for non-strings', () => { + expect(messageTemplate(null)).toBe(''); + expect(messageTemplate(undefined)).toBe(''); + expect(messageTemplate(123)).toBe(''); + }); + + it('templateOf falls back to generic mask when no override exists', () => { + expect(templateOf('UnknownFilter', "Unknown filter 'foo'")) + .toBe(messageTemplate("Unknown filter 'foo'")); + }); +}); + +describe('diagnostic-record: fingerprint hashing', () => { + it('fingerprint is stable for same (check, file, template)', () => { + const a = fingerprint('MissingPartial', 'app/x.liquid', "Cannot find "); + const b = fingerprint('MissingPartial', 'app/x.liquid', "Cannot find "); + expect(a).toBe(b); + }); + + it('fingerprint differs on file change', () => { + const a = fingerprint('MissingPartial', 'app/a.liquid', " not found"); + const b = fingerprint('MissingPartial', 'app/b.liquid', " not found"); + expect(a).not.toBe(b); + }); + + it('fingerprint differs on check change', () => { + const a = fingerprint('MissingPartial', 'app/x.liquid', ""); + const b = fingerprint('UnknownFilter', 'app/x.liquid', ""); + expect(a).not.toBe(b); + }); + + it('templateFingerprint ignores file path entirely', () => { + const a = templateFingerprint('MissingPartial', ' not found'); + const b = templateFingerprint('MissingPartial', ' not found'); + expect(a).toBe(b); + }); + + it('templateFingerprint differs from fingerprint', () => { + const tpl = ' not found'; + expect(templateFingerprint('MissingPartial', tpl)) + .not.toBe(fingerprint('MissingPartial', 'a.liquid', tpl)); + }); +}); + +describe('diagnostic-record: extractParams per check', () => { + it('UnknownFilter: pulls filter name', () => { + expect(extractParams('UnknownFilter', "Unknown filter 'json'")) + .toEqual({ filter: 'json' }); + }); + + it('UnknownFilter: empty when no quoted name', () => { + expect(extractParams('UnknownFilter', 'Unknown filter')).toEqual({}); + }); + + it('UndefinedObject: pulls variable name', () => { + expect(extractParams('UndefinedObject', "Variable 'product' is undefined")) + .toEqual({ variable: 'product' }); + }); + + it('UnusedAssign: pulls variable name', () => { + expect(extractParams('UnusedAssign', "The variable 'x' is assigned but not used")) + .toEqual({ variable: 'x' }); + }); + + it('MissingPartial: pulls partial name', () => { + expect(extractParams('MissingPartial', "'forms/login' does not exist")) + .toEqual({ partial: 'forms/login' }); + }); + + it('TranslationKeyExists: pulls key + flags typo suggestion', () => { + expect(extractParams('TranslationKeyExists', "Translation key 'a.b.c' not found. Did you mean 'a.b.cd'?")) + .toEqual({ key: 'a.b.c', has_typo_suggestion: 'true' }); + }); + + it('UnknownProperty: pulls property and object', () => { + expect(extractParams('UnknownProperty', "Unknown property `name` on `current_user`")) + .toEqual({ property: 'name', object: 'current_user' }); + }); + + it('DeprecatedTag: pulls tag and replacement', () => { + expect(extractParams('DeprecatedTag', "Tag 'include' is deprecated, use 'render'")) + .toEqual({ tag: 'include', replacement: 'render' }); + }); + + it('DeprecatedTag: include defaults replacement to render', () => { + expect(extractParams('DeprecatedTag', "'include' is deprecated")) + .toEqual({ tag: 'include', replacement: 'render' }); + }); + + it('MissingRenderPartialArguments: pulls partial + missing param', () => { + expect(extractParams('MissingRenderPartialArguments', + "Missing required argument 'email' in render tag for partial 'sessions/form'")) + .toEqual({ partial: 'sessions/form', missing_param: 'email' }); + }); + + it('MetadataParamsCheck: classifies function vs render', () => { + expect(extractParams('MetadataParamsCheck', 'Missing param in function call')) + .toEqual({ is_function_call: 'true' }); + expect(extractParams('MetadataParamsCheck', 'Missing param in render tag')) + .toEqual({ is_function_call: 'false' }); + }); + + it('GraphQLCheck: unused variable', () => { + expect(extractParams('GraphQLCheck', 'Variable "$id" is never used in operation "x"')) + .toEqual({ category: 'unused_variable', variable: 'id' }); + }); + + it('GraphQLCheck: unknown field on Record', () => { + expect(extractParams('GraphQLCheck', 'Cannot query field "name" on type "Record"')) + .toEqual({ category: 'unknown_field_record', field: 'name', type: 'Record' }); + }); + + it('GraphQLCheck: unknown field on other type', () => { + expect(extractParams('GraphQLCheck', 'Cannot query field "foo" on type "Bar"')) + .toEqual({ category: 'unknown_field_other', field: 'foo', type: 'Bar' }); + }); + + it('GraphQLCheck: type mismatch (filter)', () => { + expect(extractParams('GraphQLCheck', + 'Variable "$id" of type "ID!" used in position expecting type "UniqIdFilter"')) + .toEqual({ + category: 'type_mismatch_filter', + variable: 'id', + actual_type: 'ID!', + expected_type: 'UniqIdFilter', + }); + }); + + it('GraphQLCheck: generic fallback for unrecognized format', () => { + expect(extractParams('GraphQLCheck', 'Some unknown graphql error')) + .toEqual({ category: 'generic' }); + }); + + it('returns {} for an unknown check', () => { + expect(extractParams('NotARealCheck', 'whatever')).toEqual({}); + }); + + it('exposes the registry of known checks', () => { + expect(KNOWN_EXTRACTOR_CHECKS).toContain('MissingPartial'); + expect(KNOWN_EXTRACTOR_CHECKS).toContain('GraphQLCheck'); + expect(KNOWN_EXTRACTOR_CHECKS.length).toBeGreaterThan(5); + }); +}); + +describe('diagnostic-record: makeDiagnosticRecord', () => { + const RAW = { + check: 'MissingPartial', + severity: 'error', + message: "'forms/login' does not exist", + line: 12, + column: 4, + }; + + it('builds a frozen record with v + fp + template_fp + params', () => { + const r = makeDiagnosticRecord(RAW, { file: 'app/views/pages/x.liquid', source: 'lsp' }); + expect(r.v).toBe(DIAGNOSTIC_RECORD_VERSION); + expect(r.check).toBe('MissingPartial'); + expect(r.severity).toBe('error'); + expect(r.file).toBe('app/views/pages/x.liquid'); + expect(r.source).toBe('lsp'); + expect(r.message).toBe(RAW.message); + expect(r.message_template).toBe(' does not exist'); + expect(r.params).toEqual({ partial: 'forms/login' }); + expect(typeof r.fp).toBe('string'); + expect(r.fp).toHaveLength(40); // sha1 hex + expect(typeof r.template_fp).toBe('string'); + expect(r.template_fp).toHaveLength(40); + expect(Object.isFrozen(r)).toBe(true); + expect(Object.isFrozen(r.position)).toBe(true); + expect(Object.isFrozen(r.params)).toBe(true); + }); + + it('two records on the same (check, file, template) share fp', () => { + const a = makeDiagnosticRecord(RAW, { file: 'a.liquid', source: 'lsp' }); + const b = makeDiagnosticRecord( + { ...RAW, message: "'forms/signup' does not exist" }, // different identifier, same template + { file: 'a.liquid', source: 'lsp' }, + ); + expect(a.fp).toBe(b.fp); + expect(a.template_fp).toBe(b.template_fp); + }); + + it('records on different files share template_fp but not fp', () => { + const a = makeDiagnosticRecord(RAW, { file: 'a.liquid', source: 'lsp' }); + const b = makeDiagnosticRecord(RAW, { file: 'b.liquid', source: 'lsp' }); + expect(a.fp).not.toBe(b.fp); + expect(a.template_fp).toBe(b.template_fp); + }); + + it('position falls back to 0 when raw has no line/column', () => { + const r = makeDiagnosticRecord( + { check: 'MissingPartial', severity: 'error', message: "" }, + { file: 'x.liquid', source: 'lsp' }, + ); + expect(r.position).toEqual({ line: 0, character: 0, end_line: 0, end_character: 0 }); + }); + + it('honors explicit end_line / end_character', () => { + const r = makeDiagnosticRecord( + { ...RAW, end_line: 14, end_character: 22 }, + { file: 'x.liquid', source: 'lsp' }, + ); + expect(r.position.end_line).toBe(14); + expect(r.position.end_character).toBe(22); + }); + + it('normalizes numeric LSP severity codes', () => { + const err = makeDiagnosticRecord({ ...RAW, severity: 1 }, { file: 'x', source: 'lsp' }); + const warn = makeDiagnosticRecord({ ...RAW, severity: 2 }, { file: 'x', source: 'lsp' }); + const info = makeDiagnosticRecord({ ...RAW, severity: 3 }, { file: 'x', source: 'lsp' }); + expect(err.severity).toBe('error'); + expect(warn.severity).toBe('warning'); + expect(info.severity).toBe('info'); + }); + + it('throws on missing required fields', () => { + expect(() => makeDiagnosticRecord(null, { file: 'x', source: 'lsp' })).toThrow(); + expect(() => makeDiagnosticRecord({}, { file: 'x', source: 'lsp' })).toThrow(/check/); + expect(() => makeDiagnosticRecord(RAW, { source: 'lsp' })).toThrow(/file/); + expect(() => makeDiagnosticRecord(RAW, { file: 'x' })).toThrow(/source/); + }); + + it('records origin when supplied', () => { + const r = makeDiagnosticRecord(RAW, { + file: 'x', source: 'lsp', + origin: { check_runner_version: '4.5.5', lsp_version: '4.5.5' }, + }); + expect(r.origin).toEqual({ check_runner_version: '4.5.5', lsp_version: '4.5.5' }); + }); +}); diff --git a/tests/unit/disabled-rules.test.js b/tests/unit/disabled-rules.test.js new file mode 100644 index 0000000..d948c8f --- /dev/null +++ b/tests/unit/disabled-rules.test.js @@ -0,0 +1,93 @@ +import { describe, it, expect, beforeEach, afterEach } from 'bun:test'; +import { + registerRule, registerRules, clearRules, runRules, + updateDisabledRules, getDisabledRules, ruleCount, +} from '../../src/core/rules/engine.js'; + +function makeRule(id, check, priority = 50) { + return { + id, + check, + priority, + when: () => true, + apply: () => ({ rule_id: id, hint_md: `Hint from ${id}`, fixes: [], confidence: 0.5 }), + }; +} + +describe('J4: disabled rule enforcement', () => { + beforeEach(() => { clearRules(); updateDisabledRules([]); }); + afterEach(() => { clearRules(); updateDisabledRules([]); }); + + it('updateDisabledRules sets the disabled set', () => { + updateDisabledRules(['rule_a', 'rule_b']); + const disabled = getDisabledRules(); + expect(disabled.has('rule_a')).toBe(true); + expect(disabled.has('rule_b')).toBe(true); + expect(disabled.size).toBe(2); + }); + + it('updateDisabledRules replaces previous set', () => { + updateDisabledRules(['rule_a']); + updateDisabledRules(['rule_b']); + const disabled = getDisabledRules(); + expect(disabled.has('rule_a')).toBe(false); + expect(disabled.has('rule_b')).toBe(true); + }); + + it('updateDisabledRules with null clears all', () => { + updateDisabledRules(['rule_a']); + updateDisabledRules(null); + expect(getDisabledRules().size).toBe(0); + }); + + it('disabled rule is skipped in single-match mode', () => { + registerRules([ + makeRule('Test.high', 'Test', 10), + makeRule('Test.low', 'Test', 100), + ]); + updateDisabledRules(['Test.high']); + + const result = runRules({ check: 'Test' }, {}); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('Test.low'); + }); + + it('disabled rule is skipped in multi-match mode', () => { + registerRules([ + makeRule('Test.a', 'Test', 10), + makeRule('Test.b', 'Test', 20), + makeRule('Test.c', 'Test', 30), + ]); + updateDisabledRules(['Test.b']); + + const results = runRules({ check: 'Test' }, {}, { multiMatch: true }); + expect(results).toHaveLength(2); + expect(results.map(r => r.rule_id)).toEqual(['Test.a', 'Test.c']); + }); + + it('returns null when all rules for a check are disabled', () => { + registerRule(makeRule('Test.only', 'Test', 10)); + updateDisabledRules(['Test.only']); + + const result = runRules({ check: 'Test' }, {}); + expect(result).toBeNull(); + }); + + it('non-disabled rules fire normally', () => { + registerRule(makeRule('Test.active', 'Test', 10)); + updateDisabledRules(['SomeOther.rule']); + + const result = runRules({ check: 'Test' }, {}); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('Test.active'); + }); + + it('clearing disabled set re-enables rules', () => { + registerRule(makeRule('Test.rule', 'Test', 10)); + updateDisabledRules(['Test.rule']); + expect(runRules({ check: 'Test' }, {})).toBeNull(); + + updateDisabledRules([]); + expect(runRules({ check: 'Test' }, {})).not.toBeNull(); + }); +}); diff --git a/tests/unit/engine-mode.test.js b/tests/unit/engine-mode.test.js new file mode 100644 index 0000000..b37123b --- /dev/null +++ b/tests/unit/engine-mode.test.js @@ -0,0 +1,179 @@ +import { describe, it, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdtempSync, rmSync, readFileSync, existsSync, mkdirSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { + getEngineMode, setEngineMode, isAdaptive, + loadEngineMode, persistEngineMode, resetEngineMode, + onEngineModeChange, +} from '../../src/core/engine-mode.js'; +import { + registerRule, clearRules, runRules, + updateDisabledRules, getDisabledRules, +} from '../../src/core/rules/engine.js'; + +let tmpDir; + +beforeEach(() => { + resetEngineMode(); + clearRules(); + updateDisabledRules([]); + tmpDir = mkdtempSync(join(tmpdir(), 'engine-mode-test-')); + mkdirSync(join(tmpDir, '.pos-supervisor'), { recursive: true }); +}); + +afterEach(() => { + resetEngineMode(); + clearRules(); + updateDisabledRules([]); + rmSync(tmpDir, { recursive: true, force: true }); +}); + +describe('engine mode: core', () => { + it('defaults to static', () => { + expect(getEngineMode()).toBe('static'); + expect(isAdaptive()).toBe(false); + }); + + it('setEngineMode switches to adaptive', () => { + setEngineMode('adaptive'); + expect(getEngineMode()).toBe('adaptive'); + expect(isAdaptive()).toBe(true); + }); + + it('setEngineMode switches back to static', () => { + setEngineMode('adaptive'); + setEngineMode('static'); + expect(getEngineMode()).toBe('static'); + expect(isAdaptive()).toBe(false); + }); + + it('setEngineMode rejects invalid mode', () => { + expect(() => setEngineMode('turbo')).toThrow(/Invalid engine mode/); + }); + + it('setEngineMode is a no-op when mode unchanged', () => { + let callCount = 0; + onEngineModeChange(() => callCount++); + setEngineMode('static'); + expect(callCount).toBe(0); + }); +}); + +describe('engine mode: persistence', () => { + it('persistEngineMode writes JSON file', () => { + persistEngineMode(tmpDir, 'adaptive'); + const raw = JSON.parse(readFileSync(join(tmpDir, '.pos-supervisor', 'engine-mode.json'), 'utf-8')); + expect(raw.mode).toBe('adaptive'); + expect(raw.updated_at).toBeDefined(); + }); + + it('loadEngineMode reads from disk', () => { + persistEngineMode(tmpDir, 'adaptive'); + resetEngineMode(); + expect(getEngineMode()).toBe('static'); + + const mode = loadEngineMode(tmpDir); + expect(mode).toBe('adaptive'); + expect(getEngineMode()).toBe('adaptive'); + }); + + it('loadEngineMode returns static when file missing', () => { + const emptyDir = mkdtempSync(join(tmpdir(), 'engine-mode-empty-')); + const mode = loadEngineMode(emptyDir); + expect(mode).toBe('static'); + rmSync(emptyDir, { recursive: true, force: true }); + }); + + it('setEngineMode with projectDir persists to disk', () => { + setEngineMode('adaptive', { projectDir: tmpDir }); + const raw = JSON.parse(readFileSync(join(tmpDir, '.pos-supervisor', 'engine-mode.json'), 'utf-8')); + expect(raw.mode).toBe('adaptive'); + }); +}); + +describe('engine mode: listeners', () => { + it('onEngineModeChange fires on transition', () => { + const calls = []; + onEngineModeChange((mode, prev) => calls.push({ mode, prev })); + + setEngineMode('adaptive'); + setEngineMode('static'); + + expect(calls).toEqual([ + { mode: 'adaptive', prev: 'static' }, + { mode: 'static', prev: 'adaptive' }, + ]); + }); + + it('unsubscribe stops listener', () => { + let callCount = 0; + const unsub = onEngineModeChange(() => callCount++); + + setEngineMode('adaptive'); + expect(callCount).toBe(1); + + unsub(); + setEngineMode('static'); + expect(callCount).toBe(1); + }); + + it('listener errors are non-fatal', () => { + onEngineModeChange(() => { throw new Error('boom'); }); + expect(() => setEngineMode('adaptive')).not.toThrow(); + expect(getEngineMode()).toBe('adaptive'); + }); +}); + +describe('engine mode: onTransition callback', () => { + it('fires onTransition with prev and new mode', () => { + const transitions = []; + setEngineMode('adaptive', { + onTransition: (prev, mode) => transitions.push({ prev, mode }), + }); + expect(transitions).toEqual([{ prev: 'static', mode: 'adaptive' }]); + }); +}); + +describe('engine mode: case-base scoring gate', () => { + function makeRule(id) { + return { + id, + check: 'Test', + priority: 10, + when: () => true, + apply: () => ({ rule_id: id, hint_md: 'test', fixes: [], confidence: 0.5 }), + }; + } + + it('static mode skips case-base scoring (confidence unchanged)', () => { + registerRule(makeRule('Test.rule')); + const mockStore = { + queryOne: () => ({ emitted: 100 }), + query: () => [{ outcome: 'resolved', cnt: 90 }], + }; + + const result = runRules( + { check: 'Test', template_fp: 'abc123' }, + { analyticsStore: mockStore }, + ); + expect(result.confidence).toBe(0.5); + expect(result.case_base_signal).toBeUndefined(); + }); + + it('adaptive mode applies case-base scoring', () => { + setEngineMode('adaptive'); + registerRule(makeRule('Test.rule')); + const mockStore = { + queryOne: () => ({ emitted: 100 }), + query: () => [{ outcome: 'resolved', cnt: 90 }, { outcome: 'regressed', cnt: 5 }], + }; + + const result = runRules( + { check: 'Test', template_fp: 'abc123' }, + { analyticsStore: mockStore }, + ); + expect(result.confidence).toBeGreaterThan(0.5); + expect(result.case_base_signal).toBeDefined(); + }); +}); diff --git a/tests/unit/error-enricher-bridge.test.js b/tests/unit/error-enricher-bridge.test.js new file mode 100644 index 0000000..e4ce8c4 --- /dev/null +++ b/tests/unit/error-enricher-bridge.test.js @@ -0,0 +1,148 @@ +/** + * Bridge rules onto late-push diagnostics (2026-04-24 fix). + * + * Structural warnings, schema validators, diff-aware checks, and the + * new-partial caller check are pushed into `result.errors/warnings` AFTER + * `enrichAll` returns. Their rule modules never fire unless something runs + * the engine on them again. `bridgeRulesOntoUnattributed()` is that bridge. + */ + +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { + clearRules, registerRule, registerRules, updateForceOverrides, +} from '../../src/core/rules/engine.js'; +import { bridgeRulesOntoUnattributed } from '../../src/core/error-enricher.js'; +import { rules as NonGetRenderingPageRules } from '../../src/core/rules/NonGetRenderingPage.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; + +function resetEngine() { + clearRules(); + updateForceOverrides({ force_enable: [], force_disable: [] }); +} + +beforeEach(resetEngine); +afterEach(resetEngine); + +function emptyProjectMap() { + return { pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [] }; +} + +const ctx = { + filePath: 'app/views/pages/x.liquid', + content: '', + factGraph: buildFactGraph(emptyProjectMap()), + filtersIndex: { loaded: true, lookup: () => null, closestMatch: () => null }, + objectsIndex: { loaded: true, lookup: () => null }, + tagsIndex: { isTag: () => false }, + schemaIndex: null, + analyticsStore: null, +}; + +describe('bridgeRulesOntoUnattributed', () => { + // Task-4 split NonGetRenderingPage into three subrules + // (html_on_post / api_renders_html / get_form_target). The bridge must + // pick the subrule whose discriminator regex matches the structural + // emit's message — not the catch-all default. + test('applies registered rule to a structural diagnostic with no prior rule_id', () => { + registerRules(NonGetRenderingPageRules); + const result = { + errors: [], + warnings: [{ + check: 'pos-supervisor:NonGetRenderingPage', + severity: 'warning', + message: 'Page has `method: post` but renders HTML (layout, partials, or `{{ ... }}` output).', + line: 1, + }], + infos: [], + }; + bridgeRulesOntoUnattributed(result, ctx); + const w = result.warnings[0]; + expect(w.rule_id).toBe('NonGetRenderingPage.html_on_post'); + expect(w.confidence).toBe(0.9); + expect(w.hint).toMatch(/method: post/i); + }); + + test('skips diagnostics that already carry a rule_id (idempotent)', () => { + registerRules(NonGetRenderingPageRules); + const result = { + errors: [], + warnings: [{ + check: 'pos-supervisor:NonGetRenderingPage', + severity: 'warning', + message: 'already stamped', + rule_id: 'explicit.override', + hint: 'explicit hint', + }], + infos: [], + }; + bridgeRulesOntoUnattributed(result, ctx); + // Pre-set rule_id preserved. + expect(result.warnings[0].rule_id).toBe('explicit.override'); + expect(result.warnings[0].hint).toBe('explicit hint'); + }); + + test('no-op when check has no registered rule module', () => { + // Rule module NOT registered. Diagnostic stays unattributed; stampDefaultsOn + // in the validate-code pipeline later fills in `.unmatched` fallback. + const result = { + errors: [], + warnings: [{ + check: 'pos-supervisor:SomeCheckWithNoRule', + severity: 'warning', + message: '...', + }], + infos: [], + }; + bridgeRulesOntoUnattributed(result, ctx); + expect(result.warnings[0].rule_id).toBeUndefined(); + }); + + test('no-op when factGraph is missing (guard against partial boot)', () => { + registerRules(NonGetRenderingPageRules); + const result = { + errors: [], + warnings: [{ check: 'pos-supervisor:NonGetRenderingPage', severity: 'warning', message: '...' }], + infos: [], + }; + bridgeRulesOntoUnattributed(result, { ...ctx, factGraph: null }); + expect(result.warnings[0].rule_id).toBeUndefined(); + }); + + test('applies to errors and infos too, not just warnings', () => { + registerRule({ + id: 'SampleRule.default', + check: 'SampleCheck', + priority: 100, + when: () => true, + apply: () => ({ rule_id: 'SampleRule.default', hint_md: 'hi', fixes: [], confidence: 0.5 }), + }); + const result = { + errors: [{ check: 'SampleCheck', severity: 'error', message: 'boom' }], + warnings: [{ check: 'SampleCheck', severity: 'warning', message: 'boom' }], + infos: [{ check: 'SampleCheck', severity: 'info', message: 'boom' }], + }; + bridgeRulesOntoUnattributed(result, ctx); + expect(result.errors[0].rule_id).toBe('SampleRule.default'); + expect(result.warnings[0].rule_id).toBe('SampleRule.default'); + expect(result.infos[0].rule_id).toBe('SampleRule.default'); + }); + + test('rule that throws does not crash the bridge (non-fatal)', () => { + registerRule({ + id: 'Explosive.default', + check: 'Explosive', + priority: 100, + when: () => true, + apply: () => { throw new Error('boom'); }, + }); + const result = { + errors: [], + warnings: [{ check: 'Explosive', severity: 'warning', message: '...' }], + infos: [], + }; + // Must not throw. + bridgeRulesOntoUnattributed(result, ctx); + // Diagnostic stays unattributed — safer than half-attributed. + expect(result.warnings[0].rule_id).toBeUndefined(); + }); +}); diff --git a/tests/unit/error-enricher.test.js b/tests/unit/error-enricher.test.js index 162f1c3..aa3b1c0 100644 --- a/tests/unit/error-enricher.test.js +++ b/tests/unit/error-enricher.test.js @@ -97,13 +97,13 @@ describe('MissingPartial hint template resolution', () => { const diagnostic = { check: 'MissingPartial', severity: 'error', - message: "Missing partial 'lib/commands/products/create'", + message: "Missing partial 'commands/products/create'", line: 3, column: 3, }; const result = await enrichError(diagnostic, { uri: 'file:///app/views/pages/test.html.liquid', - content: "---\nslug: test\n---\n{% function result = 'lib/commands/products/create', params: context.params %}", + content: "---\nslug: test\n---\n{% function result = 'commands/products/create', params: context.params %}", }); expect(result.hint).toContain('command'); @@ -116,13 +116,13 @@ describe('MissingPartial hint template resolution', () => { const diagnostic = { check: 'MissingPartial', severity: 'error', - message: "Missing partial 'lib/queries/products/search'", + message: "Missing partial 'queries/products/search'", line: 3, column: 3, }; const result = await enrichError(diagnostic, { uri: 'file:///app/views/pages/test.html.liquid', - content: "---\nslug: test\n---\n{% function result = 'lib/queries/products/search', query_params: context.params %}", + content: "---\nslug: test\n---\n{% function result = 'queries/products/search', query_params: context.params %}", }); expect(result.hint).toContain('query'); @@ -131,6 +131,38 @@ describe('MissingPartial hint template resolution', () => { expect(result.hint).not.toContain('{{'); }); + it('flags `lib/` prefix as invalid and points at the corrected path', async () => { + // Regression: the `lib/commands/X` and `lib/queries/X` forms used to be + // accepted as valid call forms in our hints/data — they aren't. The + // upstream resolver searches `app/views/partials/` and `app/lib/`, so a + // literal `lib/` prefix expands to `app/lib/lib/...` and never resolves. + // The enricher must surface this distinctly, with the corrected path + // (no phantom `app/lib/lib/...`) and a "drop the prefix" message — + // never a "create the file" message. + const diagnostic = { + check: 'MissingPartial', + severity: 'error', + message: "'lib/commands/products/create' does not exist", + line: 3, + column: 3, + }; + const result = await enrichError(diagnostic, { + uri: 'file:///app/views/pages/test.html.liquid', + content: "---\nslug: test\n---\n{% function result = 'lib/commands/products/create', params: context.params %}", + }); + + expect(result.hint).toContain('lib/commands/products/create'); + expect(result.hint).toContain('commands/products/create'); + // Corrected disk path — the single-`lib/` resolution + expect(result.hint).toContain('app/lib/commands/products/create.liquid'); + // Variant must not be the create-file template — the issue is the path + // syntax, not a missing file + expect(result.hint).not.toMatch(/STEP 2 — Create/); + // Hint must call out the prefix as invalid (the fix is to drop it) + expect(result.hint).toMatch(/lib\/[^\s]+ is not a valid path|drop the `lib\/` prefix/i); + expect(result.hint).not.toContain('{{'); + }); + it('uses module variant hint for module paths — references project_map, no create path', async () => { const diagnostic = { check: 'MissingPartial', diff --git a/tests/unit/graphql-check-rules.test.js b/tests/unit/graphql-check-rules.test.js new file mode 100644 index 0000000..b7c7af9 --- /dev/null +++ b/tests/unit/graphql-check-rules.test.js @@ -0,0 +1,187 @@ +import { describe, it, expect, beforeEach } from 'bun:test'; +import { registerRules, clearRules, runRules } from '../../src/core/rules/engine.js'; +import { rules } from '../../src/core/rules/GraphQLCheck.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; + +function buildGraphWithSchema() { + return buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, + graphql: {}, layouts: {}, translations: {}, assets: [], + schema: { + blog_post: { + path: 'app/schema/blog_post.yml', + properties: [ + { name: 'title' }, + { name: 'content' }, + { name: 'author' }, + { name: 'slug' }, + ], + }, + product: { + path: 'app/schema/product.yml', + properties: [ + { name: 'name' }, + { name: 'price' }, + { name: 'sku' }, + ], + }, + }, + }); +} + +function buildMinimalGraph() { + return buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, + graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); +} + +describe('GraphQLCheck rules', () => { + beforeEach(() => { + clearRules(); + registerRules(rules); + }); + + describe('unknown_field (priority 10)', () => { + it('matches unknown field on Record type with schema suggestions', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'unknown_field_record', field: 'titl', type: 'Record' }, + message: 'Cannot query field "titl" on type "Record"', + file: 'app/graphql/get_posts.graphql', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('GraphQLCheck.unknown_field'); + expect(result.confidence).toBe(0.85); + expect(result.hint_md).toContain('titl'); + expect(result.hint_md).toContain('title'); + expect(result.hint_md).toContain('properties'); + expect(result.see_also).toBeDefined(); + expect(result.see_also.tool).toBe('domain_guide'); + }); + + it('matches unknown field on non-Record type', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'unknown_field_other', field: 'foo', type: 'UserProfile' }, + message: 'Cannot query field "foo" on type "UserProfile"', + file: 'app/graphql/get_user.graphql', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('GraphQLCheck.unknown_field'); + expect(result.confidence).toBe(0.6); + }); + + it('lists available schema tables', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'unknown_field_record', field: 'nonexistent', type: 'Record' }, + message: 'Cannot query field "nonexistent" on type "Record"', + file: 'app/graphql/test.graphql', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.hint_md).toContain('blog_post'); + expect(result.hint_md).toContain('product'); + }); + }); + + describe('unused_variable (priority 20)', () => { + it('matches unused variable diagnostics', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'unused_variable', variable: 'limit' }, + message: 'Variable "$limit" is never used in operation "GetPosts"', + file: 'app/graphql/get_posts.graphql', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('GraphQLCheck.unused_variable'); + expect(result.confidence).toBe(0.9); + expect(result.hint_md).toContain('$limit'); + }); + }); + + describe('type_mismatch (priority 30)', () => { + it('matches filter type mismatch with platformOS guidance', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'type_mismatch_filter', variable: 'name', actual_type: 'String', expected_type: 'StringFilter' }, + message: 'Variable "$name" of type "String" used in position expecting type "StringFilter"', + file: 'app/graphql/search.graphql', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('GraphQLCheck.type_mismatch'); + expect(result.confidence).toBe(0.85); + expect(result.hint_md).toContain('StringFilter'); + expect(result.hint_md).toContain('value'); + expect(result.see_also).toBeDefined(); + }); + + it('matches non-filter type mismatch', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'type_mismatch_other', variable: 'id', actual_type: 'ID!', expected_type: 'Int' }, + message: 'Variable "$id" of type "ID!" used in position expecting type "Int"', + file: 'app/graphql/get_item.graphql', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('GraphQLCheck.type_mismatch'); + expect(result.confidence).toBe(0.7); + }); + }); + + describe('generic (priority 100)', () => { + it('matches any GraphQLCheck as fallback', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'generic' }, + message: 'Some GraphQL error', + file: 'app/graphql/query.graphql', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('GraphQLCheck.generic'); + expect(result.confidence).toBe(0.4); + expect(result.see_also).toBeDefined(); + expect(result.see_also.tool).toBe('domain_guide'); + }); + }); + + describe('priority ordering', () => { + it('unknown_field wins over generic', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'unknown_field_record', field: 'title', type: 'Record' }, + message: 'Cannot query field "title" on type "Record"', + file: 'app/graphql/test.graphql', + }; + const result = runRules(diag, { graph }); + expect(result.rule_id).toBe('GraphQLCheck.unknown_field'); + }); + + it('unused_variable wins over generic', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'GraphQLCheck', + params: { category: 'unused_variable', variable: 'x' }, + message: 'Variable "$x" is never used', + file: 'test.graphql', + }; + const result = runRules(diag, { graph }); + expect(result.rule_id).toBe('GraphQLCheck.unused_variable'); + }); + }); +}); diff --git a/tests/unit/guard-synthesis.test.js b/tests/unit/guard-synthesis.test.js new file mode 100644 index 0000000..47e71e1 --- /dev/null +++ b/tests/unit/guard-synthesis.test.js @@ -0,0 +1,301 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { openAnalyticsStore } from '../../src/core/analytics-store.js'; +import { synthesizeGuardPredicate, generateRuleTemplate } from '../../src/core/case-base.js'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +function tmpPath() { + return join(tmpdir(), `pos-guard-synth-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); +} + +function seedDiagnostics(store, rows) { + for (const d of rows) { + store.db.prepare(` + INSERT INTO diagnostics (fp, template_fp, session_id, file, check_name, severity, ts, hint_rule_id, suppressed) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) + `).run( + d.fp, d.template_fp ?? 'tpl1', d.session_id ?? 'sess-1', + d.file ?? 'app/views/pages/index.html.liquid', + d.check_name ?? 'TestCheck', d.severity ?? 'error', + d.ts ?? '2026-04-20T10:00:00Z', d.hint_rule_id ?? null, d.suppressed ?? 0, + ); + } +} + +function seedEvents(store, rows) { + for (const e of rows) { + const payload = { + fp: e.fp, + template_fp: e.template_fp ?? 'tpl1', + file: e.file ?? 'app/views/pages/index.html.liquid', + check: e.check ?? 'TestCheck', + ...(e.params && Object.keys(e.params).length > 0 ? { params: e.params } : {}), + }; + store.db.prepare(` + INSERT INTO events (session_id, kind, ts, payload) + VALUES (?, ?, ?, ?) + `).run(e.session_id ?? 'sess-1', 'validator_emit', e.ts ?? '2026-04-20T10:00:00Z', JSON.stringify(payload)); + } +} + +describe('synthesizeGuardPredicate', () => { + let store, dbPath; + + beforeEach(() => { + dbPath = tmpPath(); + store = openAnalyticsStore(dbPath); + }); + afterEach(() => { store.close(); }); + + test('returns empty when object with no data', () => { + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1'); + expect(when).toEqual({}); + }); + + test('returns empty when below minSamples', () => { + seedDiagnostics(store, [ + { fp: 'fp1', file: 'app/views/pages/a.liquid' }, + { fp: 'fp2', file: 'app/views/pages/b.liquid' }, + ]); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1', { minSamples: 5 }); + expect(when).toEqual({}); + }); + + test('infers file_type when ≥80% share type', () => { + seedDiagnostics(store, [ + { fp: 'fp1', file: 'app/views/pages/a.liquid' }, + { fp: 'fp2', file: 'app/views/pages/b.liquid' }, + { fp: 'fp3', file: 'app/views/pages/c.liquid' }, + { fp: 'fp4', file: 'app/views/pages/d.liquid' }, + { fp: 'fp5', file: 'app/views/partials/e.liquid' }, + ]); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1'); + expect(when.file_type).toBe('page'); + }); + + test('skips file_type when no dominant type', () => { + seedDiagnostics(store, [ + { fp: 'fp1', file: 'app/views/pages/a.liquid' }, + { fp: 'fp2', file: 'app/views/pages/b.liquid' }, + { fp: 'fp3', file: 'app/views/partials/c.liquid' }, + { fp: 'fp4', file: 'app/views/partials/d.liquid' }, + { fp: 'fp5', file: 'app/lib/commands/e.liquid' }, + ]); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1'); + expect(when.file_type).toBeUndefined(); + }); + + test('skips file_type=unknown even if dominant', () => { + seedDiagnostics(store, [ + { fp: 'fp1', file: 'other/a.liquid' }, + { fp: 'fp2', file: 'other/b.liquid' }, + { fp: 'fp3', file: 'other/c.liquid' }, + { fp: 'fp4', file: 'other/d.liquid' }, + { fp: 'fp5', file: 'other/e.liquid' }, + ]); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1'); + expect(when.file_type).toBeUndefined(); + }); + + test('infers param_equals when ≥90% identical', () => { + const events = []; + for (let i = 0; i < 10; i++) { + events.push({ + fp: `fp${i}`, + check: 'UndefinedObject', + params: { variable: i < 9 ? 'product' : 'collection' }, + }); + } + seedDiagnostics(store, events.map(e => ({ fp: e.fp, check_name: 'UndefinedObject' }))); + seedEvents(store, events); + + const when = synthesizeGuardPredicate(store, 'UndefinedObject', 'tpl1'); + expect(when.param_equals).toEqual({ variable: 'product' }); + }); + + test('skips param_equals when below 90% threshold', () => { + const events = []; + for (let i = 0; i < 10; i++) { + events.push({ + fp: `fp${i}`, + check: 'UndefinedObject', + params: { variable: i < 8 ? 'product' : 'collection' }, + }); + } + seedDiagnostics(store, events.map(e => ({ fp: e.fp, check_name: 'UndefinedObject' }))); + seedEvents(store, events); + + const when = synthesizeGuardPredicate(store, 'UndefinedObject', 'tpl1'); + expect(when.param_equals).toBeUndefined(); + }); + + test('infers param_startsWith when ≥80% share prefix', () => { + const events = [ + { fp: 'fp0', check: 'MissingPartial', params: { partial: 'modules/core/widget' } }, + { fp: 'fp1', check: 'MissingPartial', params: { partial: 'modules/core/header' } }, + { fp: 'fp2', check: 'MissingPartial', params: { partial: 'modules/core/footer' } }, + { fp: 'fp3', check: 'MissingPartial', params: { partial: 'modules/core/sidebar' } }, + { fp: 'fp4', check: 'MissingPartial', params: { partial: 'products/card' } }, + ]; + seedDiagnostics(store, events.map(e => ({ fp: e.fp, check_name: 'MissingPartial' }))); + seedEvents(store, events); + + const when = synthesizeGuardPredicate(store, 'MissingPartial', 'tpl1'); + expect(when.param_startsWith).toBeDefined(); + expect(when.param_startsWith.partial).toBe('modules/core/'); + expect(when.param_equals).toBeUndefined(); + }); + + test('infers param_contains when ≥80% contain substring', () => { + const events = [ + { fp: 'fp0', check: 'UnknownFilter', params: { filter: 'asset_img_url' } }, + { fp: 'fp1', check: 'UnknownFilter', params: { filter: 'product_img_url' } }, + { fp: 'fp2', check: 'UnknownFilter', params: { filter: 'collection_img_url' } }, + { fp: 'fp3', check: 'UnknownFilter', params: { filter: 'variant_img_url' } }, + { fp: 'fp4', check: 'UnknownFilter', params: { filter: 'img_url' } }, + ]; + seedDiagnostics(store, events.map(e => ({ fp: e.fp, check_name: 'UnknownFilter' }))); + seedEvents(store, events); + + const when = synthesizeGuardPredicate(store, 'UnknownFilter', 'tpl1'); + expect(when.param_contains).toBeDefined(); + expect(when.param_contains.filter).toBe('img_url'); + }); + + test('combines file_type and param guards', () => { + const events = []; + for (let i = 0; i < 6; i++) { + events.push({ + fp: `fp${i}`, + check: 'UndefinedObject', + file: `app/views/pages/page${i}.liquid`, + params: { variable: 'product' }, + }); + } + seedDiagnostics(store, events.map(e => ({ fp: e.fp, check_name: 'UndefinedObject', file: e.file }))); + seedEvents(store, events); + + const when = synthesizeGuardPredicate(store, 'UndefinedObject', 'tpl1'); + expect(when.file_type).toBe('page'); + expect(when.param_equals).toEqual({ variable: 'product' }); + }); + + test('works with file_type only when no params in events', () => { + seedDiagnostics(store, [ + { fp: 'fp1', file: 'app/views/partials/a.liquid' }, + { fp: 'fp2', file: 'app/views/partials/b.liquid' }, + { fp: 'fp3', file: 'app/views/partials/c.liquid' }, + { fp: 'fp4', file: 'app/views/partials/d.liquid' }, + { fp: 'fp5', file: 'app/views/partials/e.liquid' }, + ]); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1'); + expect(when.file_type).toBe('partial'); + expect(when.param_equals).toBeUndefined(); + expect(when.param_startsWith).toBeUndefined(); + expect(when.param_contains).toBeUndefined(); + }); + + test('respects minSamples for param analysis', () => { + const events = [ + { fp: 'fp0', check: 'TestCheck', params: { key: 'same' } }, + { fp: 'fp1', check: 'TestCheck', params: { key: 'same' } }, + { fp: 'fp2', check: 'TestCheck', params: { key: 'same' } }, + ]; + seedDiagnostics(store, events.map(e => ({ fp: e.fp }))); + seedEvents(store, events); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1', { minSamples: 5 }); + expect(when.param_equals).toBeUndefined(); + }); + + test('ignores suppressed diagnostics for file_type', () => { + seedDiagnostics(store, [ + { fp: 'fp1', file: 'app/views/pages/a.liquid', suppressed: 0 }, + { fp: 'fp2', file: 'app/views/pages/b.liquid', suppressed: 0 }, + { fp: 'fp3', file: 'app/views/pages/c.liquid', suppressed: 0 }, + { fp: 'fp4', file: 'app/views/pages/d.liquid', suppressed: 0 }, + { fp: 'fp5', file: 'app/views/pages/e.liquid', suppressed: 0 }, + { fp: 'fp6', file: 'app/lib/commands/x.liquid', suppressed: 1 }, + { fp: 'fp7', file: 'app/lib/commands/y.liquid', suppressed: 1 }, + { fp: 'fp8', file: 'app/lib/commands/z.liquid', suppressed: 1 }, + ]); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1'); + expect(when.file_type).toBe('page'); + }); + + test('prefers param_equals over param_startsWith', () => { + const events = []; + for (let i = 0; i < 10; i++) { + events.push({ fp: `fp${i}`, check: 'TestCheck', params: { key: 'exactly_the_same' } }); + } + seedDiagnostics(store, events.map(e => ({ fp: e.fp }))); + seedEvents(store, events); + + const when = synthesizeGuardPredicate(store, 'TestCheck', 'tpl1'); + expect(when.param_equals).toEqual({ key: 'exactly_the_same' }); + expect(when.param_startsWith).toBeUndefined(); + expect(when.param_contains).toBeUndefined(); + }); +}); + +describe('generateRuleTemplate with guards', () => { + const baseSuggestion = { + check: 'UnknownFilter', + template_fp: 'abcdef1234567890', + resolution_rate: 0.85, + total_outcomes: 20, + sample_file: 'app/views/partials/test.liquid', + }; + + test('renders TODO when no guards', () => { + const template = generateRuleTemplate(baseSuggestion); + expect(template).toContain('TODO: Add guard predicate'); + expect(template).toContain('return true;'); + }); + + test('renders file_type guard', () => { + const template = generateRuleTemplate(baseSuggestion, { file_type: 'page' }); + expect(template).toContain("diag.file?.includes(\"/pages/\")"); + expect(template).not.toContain('TODO: Add guard predicate'); + }); + + test('renders param_equals guard', () => { + const template = generateRuleTemplate(baseSuggestion, { param_equals: { filter: 'asset_url' } }); + expect(template).toContain("diag.params?.filter === \"asset_url\""); + }); + + test('renders param_startsWith guard', () => { + const template = generateRuleTemplate(baseSuggestion, { param_startsWith: { partial: 'modules/' } }); + expect(template).toContain("diag.params?.partial?.startsWith(\"modules/\")"); + }); + + test('renders param_contains guard', () => { + const template = generateRuleTemplate(baseSuggestion, { param_contains: { filter: 'img_url' } }); + expect(template).toContain("diag.params?.filter?.includes(\"img_url\")"); + }); + + test('renders combined guards with &&', () => { + const guards = { + file_type: 'partial', + param_equals: { variable: 'product' }, + }; + const template = generateRuleTemplate(baseSuggestion, guards); + expect(template).toContain('&&'); + expect(template).toContain("diag.params?.variable === \"product\""); + expect(template).toContain("diag.file?.includes(\"/partials/\")"); + }); + + test('preserves existing template fields', () => { + const template = generateRuleTemplate(baseSuggestion, { file_type: 'page' }); + expect(template).toContain("id: 'UnknownFilter.case_abcdef12'"); + expect(template).toContain("check: 'UnknownFilter'"); + expect(template).toContain('confidence: 0.85'); + expect(template).toContain('85% across 20 outcomes'); + }); +}); diff --git a/tests/unit/health-scores.test.js b/tests/unit/health-scores.test.js new file mode 100644 index 0000000..d353c07 --- /dev/null +++ b/tests/unit/health-scores.test.js @@ -0,0 +1,87 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { openAnalyticsStore } from '../../src/core/analytics-store.js'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +function tmpPath() { + return join(tmpdir(), `pos-health-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); +} + +describe('K1: health score history', () => { + let store; + beforeEach(() => { store = openAnalyticsStore(tmpPath()); }); + afterEach(() => { store.close(); }); + + test('insertHealthScore and getHealthScores round-trip', () => { + store.insertHealthScore({ + score: 72, + mode: 'project', + dimensions: { errors: 90, warnings: 80, orphaned: 65, coverage: 55 }, + }); + + const scores = store.getHealthScores({ limit: 10 }); + expect(scores).toHaveLength(1); + expect(scores[0].score).toBe(72); + expect(scores[0].mode).toBe('project'); + expect(scores[0].dimensions).toEqual({ errors: 90, warnings: 80, orphaned: 65, coverage: 55 }); + expect(scores[0].ts).toBeTruthy(); + }); + + test('getHealthScores returns chronological order', () => { + store.insertHealthScore({ score: 50, mode: 'project', dimensions: {} }); + store.insertHealthScore({ score: 60, mode: 'project', dimensions: {} }); + store.insertHealthScore({ score: 70, mode: 'project', dimensions: {} }); + + const scores = store.getHealthScores({ limit: 10 }); + expect(scores).toHaveLength(3); + expect(scores[0].score).toBe(50); + expect(scores[1].score).toBe(60); + expect(scores[2].score).toBe(70); + }); + + test('getHealthScores respects limit', () => { + for (let i = 0; i < 10; i++) { + store.insertHealthScore({ score: i * 10, mode: 'project', dimensions: {} }); + } + + const scores = store.getHealthScores({ limit: 3 }); + expect(scores).toHaveLength(3); + expect(scores[0].score).toBe(70); + expect(scores[2].score).toBe(90); + }); + + test('getHealthScores filters by mode', () => { + store.insertHealthScore({ score: 50, mode: 'project', dimensions: {} }); + store.insertHealthScore({ score: 80, mode: 'infrastructure', dimensions: {} }); + store.insertHealthScore({ score: 60, mode: 'project', dimensions: {} }); + + const projectScores = store.getHealthScores({ mode: 'project' }); + expect(projectScores).toHaveLength(2); + expect(projectScores.every(s => s.mode === 'project')).toBe(true); + + const infraScores = store.getHealthScores({ mode: 'infrastructure' }); + expect(infraScores).toHaveLength(1); + expect(infraScores[0].score).toBe(80); + }); + + test('getHealthScores returns empty for no data', () => { + const scores = store.getHealthScores(); + expect(scores).toEqual([]); + }); + + test('health_scores table survives rebuild', () => { + store.insertHealthScore({ score: 75, mode: 'project', dimensions: { x: 1 } }); + + const { mkdirSync } = require('node:fs'); + const sessionsDir = join(tmpdir(), `pos-health-sessions-${Date.now()}`); + mkdirSync(sessionsDir, { recursive: true }); + store.rebuild(sessionsDir); + + const scores = store.getHealthScores(); + expect(scores).toHaveLength(1); + expect(scores[0].score).toBe(75); + + const { rmSync } = require('node:fs'); + try { rmSync(sessionsDir, { recursive: true }); } catch {} + }); +}); diff --git a/tests/unit/http-handler-arity.test.js b/tests/unit/http-handler-arity.test.js new file mode 100644 index 0000000..517b039 --- /dev/null +++ b/tests/unit/http-handler-arity.test.js @@ -0,0 +1,158 @@ +/** + * Static guard against the "caller passes the wrong number of args to an + * analytics handler" class of bug. + * + * Phase 5 (`since` parameter wiring) widened the signatures of three + * analytics handlers (`handleFixAdoptionFunnel`, `handleKnowledgeGaps`, + * `handleRuleHeatmap`) from `(store, res)` to `(store, url, res)`. Two + * other handlers were never widened during a follow-up review + * (`handleAnalyticsSessions`, `handleSuggestedRules`), and the matching + * call sites silently passed `(store, res)`. Bun runtime then evaluates + * `sendJson(res, ...)` with `res === undefined`, which throws a + * `TypeError: undefined is not an object (evaluating 'res.writeHead')` — + * the entire HTTP listener dies inside the request handler, leaving the + * MCP stdio process alive but the dashboard offline. + * + * The unit-test surface for that bug is awkward (handlers take real + * `req`/`res` streams). A static-source check is cheap, deterministic, + * and covers every analytics handler at once: read http-server.js as + * text, extract every handler's parameter count from its declaration, + * extract every call site's argument count from `return handleX(...)`, + * and assert they match. + * + * If you add a new handler, you don't need to touch this test — it + * discovers handlers + call sites by pattern. + */ + +import { describe, test, expect } from 'bun:test'; +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; + +const SRC = readFileSync( + join(import.meta.dir, '..', '..', 'src', 'http-server.js'), + 'utf8', +); + +/** + * Parse "function handleX(a, b, c) { ... }" declarations. + * Returns Map. Async-function declarations + * are supported; rest params (...x) are intentionally counted as 1 + * because we don't dispatch with spread syntax. + */ +function extractHandlerArities(src) { + const re = /function\s+(handle[A-Za-z]+)\s*\(([^)]*)\)/g; + const out = new Map(); + let m; + while ((m = re.exec(src))) { + const name = m[1]; + const paramList = m[2].trim(); + const arity = paramList === '' ? 0 : paramList.split(',').filter(Boolean).length; + out.set(name, arity); + } + return out; +} + +/** + * Parse "return handleX(a, b, c)" call sites. Tolerates whitespace and + * matches up to the closing paren on the same line — every analytics + * dispatch is a single-line return. + */ +function extractCallSites(src) { + const re = /return\s+(handle[A-Za-z]+)\s*\(([^)]*)\)/g; + const out = []; + let m; + while ((m = re.exec(src))) { + const name = m[1]; + const argList = m[2].trim(); + const arity = argList === '' ? 0 : argList.split(',').filter(Boolean).length; + // Compute the source line for nicer test failures. + const line = src.slice(0, m.index).split('\n').length; + out.push({ name, arity, line }); + } + return out; +} + +describe('http-server.js handler dispatch arity', () => { + const arities = extractHandlerArities(SRC); + const callSites = extractCallSites(SRC); + + test('handler declarations are discovered', () => { + // Sanity check the parser — http-server is large enough that we + // expect dozens of analytics handlers. If this drops to zero, the + // regex broke and the rest of the suite is silently green-on-zero. + expect(arities.size).toBeGreaterThan(20); + }); + + test('every analytics call site uses the declared arity', () => { + // Restrict to analytics handlers — the other handlers in this file + // have varied signatures (some take projectDir, some take body, + // etc.) and aren't part of the Phase 5 surface this guard protects. + const ANALYTICS_PREFIXES = [ + 'handleAnalytics', + 'handleRule', + 'handleFixRule', + 'handleConfidence', + 'handleFixAdoption', + 'handleKnowledge', + 'handleDiagnostic', + 'handleSuggested', + 'handleCases', + ]; + const isAnalyticsHandler = (name) => + ANALYTICS_PREFIXES.some(prefix => name.startsWith(prefix)); + + const mismatches = []; + for (const { name, arity, line } of callSites) { + if (!isAnalyticsHandler(name)) continue; + const declared = arities.get(name); + if (declared == null) continue; // imported handler — not declared in this file + if (declared !== arity) { + mismatches.push( + `http-server.js:${line} — return ${name}(...) passes ${arity} args, but ${name}() declares ${declared} parameters`, + ); + } + } + expect(mismatches).toEqual([]); + }); + + test('every analytics handler that takes `url` actually uses it', () => { + // Belt-and-braces: catch the inverse — a handler whose signature + // declares (store, url, res) but whose body never references `url` + // is a dead parameter that probably indicates a broken refactor. + const ANALYTICS_NAMES = [...arities.keys()].filter(n => + n.startsWith('handleAnalytics') || + n.startsWith('handleRule') || + n.startsWith('handleFixRule') || + n.startsWith('handleConfidence') || + n.startsWith('handleFixAdoption') || + n.startsWith('handleKnowledge') || + n.startsWith('handleDiagnostic') || + n.startsWith('handleSuggested') || + n.startsWith('handleCases') + ); + const dead = []; + for (const name of ANALYTICS_NAMES) { + // Find the function body — slice from declaration to end of file + // and stop at the next top-level `function ` declaration. + const declRe = new RegExp(`function\\s+${name}\\s*\\(([^)]*)\\)`); + const declMatch = SRC.match(declRe); + if (!declMatch) continue; + const params = declMatch[1].split(',').map(p => p.trim()).filter(Boolean); + if (!params.includes('url')) continue; // doesn't take url — skip + + const startIdx = declMatch.index + declMatch[0].length; + const restOfFile = SRC.slice(startIdx); + // Function body ends at the next "\n}\n\n" sequence or next "function " + // declaration at column 0. The simpler heuristic: look for "\nfunction " + // and slice up to it. + const nextDeclIdx = restOfFile.search(/\nfunction\s+\w/); + const body = nextDeclIdx >= 0 ? restOfFile.slice(0, nextDeclIdx) : restOfFile; + + // Use \burl\b to avoid matching "url" as part of another identifier. + if (!/\burl\b/.test(body)) { + dead.push(`${name} declares 'url' parameter but body never references it`); + } + } + expect(dead).toEqual([]); + }); +}); diff --git a/tests/unit/http-server.test.js b/tests/unit/http-server.test.js index ae29f88..776bfba 100644 --- a/tests/unit/http-server.test.js +++ b/tests/unit/http-server.test.js @@ -96,6 +96,95 @@ describe('HTTP GET endpoints', () => { }); }); +describe('HTTP GET /api/hints', () => { + it('list mode returns both static md hints and rule-driven check names', async () => { + const { status, body } = await httpGet('/api/hints'); + expect(status).toBe(200); + expect(Array.isArray(body.hints)).toBe(true); + expect(Array.isArray(body.checks)).toBe(true); + + // Legacy md hints — sample from src/data/hints/ + expect(body.hints).toContain('GraphQLCheck'); + // Rule-driven (no md file) — must be present after the fix. + expect(body.hints).toContain('GraphQLVariablesCheck'); + expect(body.hints).toContain('PartialCallArguments'); + + // Per-check sources metadata + const gqlVars = body.checks.find(c => c.name === 'GraphQLVariablesCheck'); + expect(gqlVars).toBeDefined(); + expect(gqlVars.sources).toContain('rule'); + expect(gqlVars.sources).not.toContain('static'); + + const gqlCheck = body.checks.find(c => c.name === 'GraphQLCheck'); + expect(gqlCheck).toBeDefined(); + expect(gqlCheck.sources).toContain('static'); + }); + + it('GET ?name= returns md content with source=static', async () => { + const { status, body } = await httpGet('/api/hints?name=GraphQLCheck'); + expect(status).toBe(200); + expect(body.name).toBe('GraphQLCheck'); + expect(body.source).toBe('static'); + expect(typeof body.content).toBe('string'); + expect(body.content.length).toBeGreaterThan(20); + }); + + // Repro for the dashboard 404 reported on 2026-04-28: rule-driven checks + // had no md file, the endpoint 404'd, drilldown showed "Failed to load". + it('GET ?name=GraphQLVariablesCheck synthesizes a rule doc instead of 404', async () => { + const { status, body } = await httpGet('/api/hints?name=GraphQLVariablesCheck'); + expect(status).toBe(200); + expect(body.name).toBe('GraphQLVariablesCheck'); + expect(body.source).toBe('rule'); + expect(Array.isArray(body.rule_ids)).toBe(true); + // Sub-rule ids must be present in the synthesized doc. + expect(body.rule_ids).toContain('GraphQLVariablesCheck.required'); + expect(body.rule_ids).toContain('GraphQLVariablesCheck.unknown'); + expect(body.rule_ids).toContain('GraphQLVariablesCheck.parser_blind_spot'); + expect(body.content).toContain('GraphQLVariablesCheck'); + expect(body.content).toContain('Rule-driven'); + expect(body.content).toContain('src/core/rules/GraphQLVariablesCheck.js'); + // Each sub-rule must be documented with priority + when() source. + expect(body.content).toContain('parser_blind_spot'); + expect(body.content).toContain('priority'); + }); + + it('GET ?name= still 404s when neither md nor rule exists', async () => { + const { status, body } = await httpGet('/api/hints?name=NoSuchCheckEverDefined'); + expect(status).toBe(404); + expect(body.error).toBeDefined(); + }); + + it('GET ?name= resolves prefixed rule-driven checks', async () => { + // pos-supervisor:InvalidLayout is registered with the prefix, has no md + // file, and the dashboard splits on the first dot when computing + // baseCheck — so the colon prefix must round-trip cleanly. + const { status, body } = await httpGet( + '/api/hints?name=' + encodeURIComponent('pos-supervisor:InvalidLayout') + ); + expect(status).toBe(200); + expect(body.source).toBe('rule'); + expect(body.name).toBe('pos-supervisor:InvalidLayout'); + // Module path strips the `pos-supervisor:` prefix when we point at the + // file the developer must edit — the rule files are not namespaced. + expect(body.content).toContain('src/core/rules/InvalidLayout.js'); + }); + + it('GET ?name= with both md and rule prefers static md', async () => { + // pos-supervisor:NonGetRenderingPage has BOTH a md file and a rule + // module. The endpoint must serve the md (legacy enricher path is what + // agents actually consume) and the rule sub-rules remain reachable via + // the per-check rule_ids list elsewhere in the dashboard. + const { status, body } = await httpGet( + '/api/hints?name=' + encodeURIComponent('pos-supervisor:NonGetRenderingPage') + ); + expect(status).toBe(200); + expect(body.source).toBe('static'); + expect(typeof body.content).toBe('string'); + expect(body.content.length).toBeGreaterThan(20); + }); +}); + describe('HTTP POST /call', () => { it('executes domain_guide tool', async () => { const { status, body } = await httpPost('/call', { diff --git a/tests/unit/http-since-param.test.js b/tests/unit/http-since-param.test.js new file mode 100644 index 0000000..c3d2fff --- /dev/null +++ b/tests/unit/http-since-param.test.js @@ -0,0 +1,63 @@ +/** + * Unit tests for `parseSinceParam` — the HTTP-side translator that maps + * the `?since=` query parameter to the tri-state contract used by + * analytics-queries + case-base reporting paths. + * + * The parser sits at the HTTP boundary. Integration tests can't cover it + * cleanly because the spawned-Node server can't open `bun:sqlite`, so the + * analytics endpoints return 503 in that environment. A unit test on the + * pure parser pins the contract every endpoint depends on. + */ + +import { describe, test, expect } from 'bun:test'; +import { parseSinceParam } from '../../src/http-server.js'; + +function urlWith(qs) { + return new URL(`http://localhost/foo${qs}`); +} + +describe('parseSinceParam tri-state contract', () => { + test('absent ?since → undefined (meta default applies)', () => { + expect(parseSinceParam(urlWith(''))).toBeUndefined(); + expect(parseSinceParam(urlWith('?other=1'))).toBeUndefined(); + }); + + test('empty ?since → undefined', () => { + expect(parseSinceParam(urlWith('?since='))).toBeUndefined(); + }); + + test('?since=all → null (engine-state bypass marker)', () => { + expect(parseSinceParam(urlWith('?since=all'))).toBeNull(); + }); + + test('?since= → the same ISO string', () => { + const ts = '2026-04-30T12:00:00.000Z'; + expect(parseSinceParam(urlWith(`?since=${encodeURIComponent(ts)}`))).toBe(ts); + }); + + test('?since= throws a 400-eligible Error', () => { + // The thrown message must include "since must be" so http-server.js's + // sinceErrorStatus() routes it as 400 rather than 500. + expect(() => parseSinceParam(urlWith('?since=not-a-date'))).toThrow(/since must be/); + }); + + test('whitespace-only ?since is rejected', () => { + // Date(' ') parses NaN → must throw. + expect(() => parseSinceParam(urlWith('?since=%20%20%20'))).toThrow(/since must be/); + }); + + test('case-sensitivity: only literal "all" is the bypass', () => { + // 'All' / 'ALL' must NOT collapse to null — strict matching avoids + // accidental bypass from typos that happen to parse as a Date elsewhere. + expect(() => parseSinceParam(urlWith('?since=All'))).toThrow(); + expect(() => parseSinceParam(urlWith('?since=ALL'))).toThrow(); + }); + + test('non-ISO but Date-parseable strings are accepted (Date is lenient)', () => { + // `new Date('2026-04-30')` parses to a valid date. The parser only + // rejects strings that fail Date — it does not enforce strict ISO 8601. + // This matches existing analytics flexibility (the SQL filter just + // compares strings); pin the behaviour so it doesn't drift. + expect(parseSinceParam(urlWith('?since=2026-04-30'))).toBe('2026-04-30'); + }); +}); diff --git a/tests/unit/liquid-parser.test.js b/tests/unit/liquid-parser.test.js index 063675b..a54df36 100644 --- a/tests/unit/liquid-parser.test.js +++ b/tests/unit/liquid-parser.test.js @@ -51,6 +51,66 @@ describe('extractAllFromAST', () => { expect(result.graphql[0].queryName).toBe('products/search'); }); + it('captures named-argument names and source_kind=tag for the canonical tag form', () => { + const ast = parseLiquidFile( + "{% graphql result = 'op', name: shaped.name, email: shaped.email %}" + ); + const result = extractAllFromAST(ast); + expect(result.graphql).toHaveLength(1); + expect(result.graphql[0].args).toEqual(['name', 'email']); + expect(result.graphql[0].source_kind).toBe('tag'); + }); + + it('classifies single-line graphql inside {% liquid %} block as liquid_inline', () => { + const ast = parseLiquidFile( + "{% liquid\ngraphql result = 'op', name: shaped.name, email: shaped.email\n%}" + ); + const result = extractAllFromAST(ast); + expect(result.graphql).toHaveLength(1); + expect(result.graphql[0].args).toEqual(['name', 'email']); + expect(result.graphql[0].source_kind).toBe('liquid_inline'); + }); + + // Repro for the DEMO regression spiral (2026-04-27): multi-line comma + // continuation inside `{% liquid %}` block. The liquid-html-parser truncates + // the call at the first newline — markup.args is empty, the args are + // silently dropped, and pos-cli's LSP fires "Required parameter X missing" + // for each. The classifier flags this so the rule layer can route to a + // syntax-fix hint instead of the misleading "add the arg" hint. + it('flags multi-line graphql in {% liquid %} block as liquid_multiline_truncated', () => { + const ast = parseLiquidFile( + "{% liquid\ngraphql result = 'op',\n name: shaped.name,\n email: shaped.email\n%}" + ); + const result = extractAllFromAST(ast); + expect(result.graphql).toHaveLength(1); + expect(result.graphql[0].source_kind).toBe('liquid_multiline_truncated'); + // Args extracted by markup.args are empty here (parser truncation) — the + // detector must not depend on args.length to distinguish the kind. + expect(result.graphql[0].args).toEqual([]); + }); + + it('does not flag a comma-ending inline call without trailing named-arg lines', () => { + // No `name:` continuation after — just a stray comma inside whatever + // followed in the liquid block. Should NOT be classified as truncated. + const ast = parseLiquidFile( + "{% liquid\ngraphql result = 'op', name: shaped.name,\nassign other = 1\n%}" + ); + const result = extractAllFromAST(ast); + expect(result.graphql).toHaveLength(1); + expect(result.graphql[0].source_kind).toBe('liquid_inline'); + }); + + it('upgrades source_kind to truncated when any duplicate call is truncated', () => { + const ast = parseLiquidFile( + "{% graphql a = 'op', name: x %}\n" + + "{% liquid\ngraphql b = 'op',\n name: y,\n email: z\n%}" + ); + const result = extractAllFromAST(ast); + // Dedup keeps a single entry but the surface kind reflects the worst case. + expect(result.graphql).toHaveLength(1); + expect(result.graphql[0].source_kind).toBe('liquid_multiline_truncated'); + }); + it('extracts filter names', () => { const ast = parseLiquidFile("{{ 'hello' | t }}\n{{ price | pricify | json }}"); const result = extractAllFromAST(ast); diff --git a/tests/unit/lsp-stale-diagnostics.test.js b/tests/unit/lsp-stale-diagnostics.test.js index 4b75a44..405d730 100644 --- a/tests/unit/lsp-stale-diagnostics.test.js +++ b/tests/unit/lsp-stale-diagnostics.test.js @@ -88,20 +88,19 @@ describe('awaitDiagnostics barrier + settle pattern', () => { client.stop(); }); - it('post-barrier diagnostics replace pre-barrier ones', async () => { + it('later diagnostics replace earlier ones via settle window', async () => { const { client, send, captured } = startClient(); const uri = 'file:///test/app/stale.liquid'; const promise = client.awaitDiagnostics(uri, 'line1\nline2\n', 2000); const hoverReq = captured.find(m => m.method === 'textDocument/hover'); - // Pre-barrier diagnostics — stored but settle not started yet - send(diagNotification(uri, [diag(9, 'PreBarrierCheck', 'from old analysis')])); + // Early diagnostics accepted, settle timer starts + send(diagNotification(uri, [diag(9, 'EarlyCheck', 'from old analysis')])); - // Barrier response — if we have diags, settle timer starts send(hoverResponse(hoverReq.id)); - // Fresh diagnostics arrive AFTER barrier — replace pre-barrier ones, reset settle + // Later diagnostics replace earlier ones, settle timer resets send(diagNotification(uri, [diag(0, 'FreshCheck', 'from new content')])); const result = await promise; @@ -110,18 +109,15 @@ describe('awaitDiagnostics barrier + settle pattern', () => { client.stop(); }); - it('handles pre-barrier + barrier + post-barrier in rapid sequence', async () => { + it('handles rapid sequence of diagnostics — settle picks latest', async () => { const { client, send, captured } = startClient(); const uri = 'file:///test/app/onechunk.liquid'; const promise = client.awaitDiagnostics(uri, 'a\nb\n', 2000); const hoverReq = captured.find(m => m.method === 'textDocument/hover'); - // Write all three messages rapidly — #drain processes them sequentially - // 1. pre-barrier diag → stored, settle deferred - // 2. hover response → barrier passes, settle starts - // 3. post-barrier diag → replaces pre-barrier, settle resets - send(diagNotification(uri, [diag(10, 'PreBarrierCheck')])); + // All three messages arrive rapidly — settle picks the last batch + send(diagNotification(uri, [diag(10, 'EarlyCheck')])); send(hoverResponse(hoverReq.id)); send(diagNotification(uri, [diag(0, 'FreshCheck')])); @@ -131,6 +127,25 @@ describe('awaitDiagnostics barrier + settle pattern', () => { client.stop(); }); + it('accepts diagnostics that arrive before barrier hover responds', async () => { + const { client, send, captured } = startClient(); + const uri = 'file:///test/app/fast-lsp.liquid'; + + const promise = client.awaitDiagnostics(uri, '{{ x | bad_filter }}\n', 2000); + const hoverReq = captured.find(m => m.method === 'textDocument/hover'); + + // LSP emits diagnostics BEFORE responding to hover (fast analysis) + send(diagNotification(uri, [diag(0, 'UnknownFilter', 'Unknown filter bad_filter')])); + + // Hover response arrives later + send(hoverResponse(hoverReq.id)); + + const result = await promise; + expect(result).toHaveLength(1); + expect(result[0].code).toBe('UnknownFilter'); + client.stop(); + }); + it('resolves with empty array on timeout (no diagnostics after barrier)', async () => { const { client, send, captured } = startClient(); const uri = 'file:///test/app/timeout.liquid'; @@ -156,18 +171,16 @@ describe('awaitDiagnostics barrier + settle pattern', () => { client.stop(); }); - it('accepts diagnostics after barrier timeout (barrier gate opens on timeout)', async () => { + it('accepts diagnostics even when barrier hover never responds', async () => { const { client, send } = startClient(); const uri = 'file:///test/app/late-barrier.liquid'; - // Use a longer main timeout so diagnostics can arrive after barrier timeout const promise = client.awaitDiagnostics(uri, 'content\n', 5000); - // Don't respond to barrier — it will time out after 3s (min of timeout, 3000) - // But diagnostics arrive after the barrier timeout + // Don't respond to barrier hover — diagnostics still accepted setTimeout(() => { send(diagNotification(uri, [diag(0, 'LateCheck')])); - }, 3100); + }, 100); const result = await promise; expect(result).toHaveLength(1); @@ -187,17 +200,14 @@ describe('awaitDiagnostics barrier + settle pattern', () => { client.stop(); }); - it('barrier gate opens on hover error response', async () => { + it('accepts diagnostics when hover responds with error', async () => { const { client, send, captured } = startClient(); const uri = 'file:///test/app/hover-error.liquid'; const promise = client.awaitDiagnostics(uri, 'content\n', 2000); const hoverReq = captured.find(m => m.method === 'textDocument/hover'); - // LSP responds with error (e.g., unsupported file type) send({ jsonrpc: '2.0', id: hoverReq.id, error: { code: -32601, message: 'not supported' } }); - - // Fresh diagnostics should still be accepted send(diagNotification(uri, [diag(0, 'FreshAfterError')])); const result = await promise; @@ -219,19 +229,18 @@ describe('awaitDiagnostics barrier + settle pattern', () => { expect(r1).toHaveLength(1); expect(r1[0].code).toBe('V1Check'); - // Second call — new barrier + // Second call — settle window picks the latest batch captured.length = 0; const p2 = client.awaitDiagnostics(uri, 'v2\n', 2000); const hover2 = captured.find(m => m.method === 'textDocument/hover'); expect(hover2).toBeDefined(); - // Pre-barrier diagnostics from v1 analysis still arriving - send(diagNotification(uri, [diag(0, 'PreBarrierV1')])); + // Stale v1 diagnostics arrive first + send(diagNotification(uri, [diag(0, 'StaleV1')])); - // Barrier passes send(hoverResponse(hover2.id)); - // Fresh v2 diagnostics replace pre-barrier ones + // Fresh v2 diagnostics replace stale ones via settle window send(diagNotification(uri, [diag(0, 'V2Check')])); const r2 = await p2; diff --git a/tests/unit/metadata-params-rules.test.js b/tests/unit/metadata-params-rules.test.js new file mode 100644 index 0000000..2c63ca8 --- /dev/null +++ b/tests/unit/metadata-params-rules.test.js @@ -0,0 +1,153 @@ +import { describe, it, expect, beforeEach } from 'bun:test'; +import { registerRules, clearRules, runRules } from '../../src/core/rules/engine.js'; +import { rules } from '../../src/core/rules/MetadataParamsCheck.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; + +function buildGraphWithPartials() { + return buildFactGraph({ + pages: {}, commands: {}, queries: {}, + graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + partials: { + 'shared/card': { + path: 'app/views/partials/shared/card.liquid', + params: [ + { name: 'title', required: true }, + { name: 'body', required: true }, + { name: 'class', required: false }, + ], + renders: [], + function_calls: [], + }, + 'layouts/header': { + path: 'app/views/partials/layouts/header.liquid', + params: [ + { name: 'logo_url', required: true }, + ], + renders: [], + function_calls: [], + }, + }, + }); +} + +function buildMinimalGraph() { + return buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, + graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); +} + +describe('MetadataParamsCheck rules', () => { + beforeEach(() => { + clearRules(); + registerRules(rules); + }); + + describe('module_contract (priority 10)', () => { + it('matches module partial paths', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'MetadataParamsCheck', + params: { is_function_call: 'false' }, + message: "Missing required parameter in 'modules/core/lib/helpers/format_date'", + file: 'app/views/pages/index.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MetadataParamsCheck.module_contract'); + expect(result.confidence).toBe(0.85); + expect(result.see_also).toBeDefined(); + expect(result.see_also.tool).toBe('module_info'); + expect(result.see_also.args.name).toBe('core'); + }); + + it('handles function call variant', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'MetadataParamsCheck', + params: { is_function_call: 'true' }, + message: "Missing required parameter in function call 'modules/user/commands/create'", + file: 'app/views/pages/signup.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MetadataParamsCheck.module_contract'); + expect(result.hint_md).toContain('function'); + }); + }); + + describe('doc_block_params (priority 20)', () => { + it('shows declared params from fact graph', () => { + const graph = buildGraphWithPartials(); + const diag = { + check: 'MetadataParamsCheck', + params: { is_function_call: 'false' }, + message: "Missing required parameter in 'shared/card'", + file: 'app/views/pages/blog.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MetadataParamsCheck.doc_block_params'); + expect(result.confidence).toBe(0.8); + expect(result.hint_md).toContain('title'); + expect(result.hint_md).toContain('required'); + expect(result.hint_md).toContain('class'); + expect(result.hint_md).toContain('optional'); + expect(result.see_also.tool).toBe('domain_guide'); + }); + + it('does not match when partial has no params', () => { + const graph = buildFactGraph({ + pages: {}, commands: {}, queries: {}, + graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + partials: { + 'simple/block': { + path: 'app/views/partials/simple/block.liquid', + params: [], + renders: [], + function_calls: [], + }, + }, + }); + const diag = { + check: 'MetadataParamsCheck', + params: { is_function_call: 'false' }, + message: "Missing required parameter in 'simple/block'", + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MetadataParamsCheck.generic'); + }); + }); + + describe('generic (priority 100)', () => { + it('matches any MetadataParamsCheck as fallback', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'MetadataParamsCheck', + params: { is_function_call: 'false' }, + message: 'Some metadata params error', + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MetadataParamsCheck.generic'); + expect(result.confidence).toBe(0.4); + expect(result.hint_md).toContain('Render call'); + }); + + it('uses function call wording when is_function_call', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'MetadataParamsCheck', + params: { is_function_call: 'true' }, + message: 'Some metadata params error in function call', + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.hint_md).toContain('Function call'); + }); + }); +}); diff --git a/tests/unit/missing-render-args-rules.test.js b/tests/unit/missing-render-args-rules.test.js new file mode 100644 index 0000000..bafde6a --- /dev/null +++ b/tests/unit/missing-render-args-rules.test.js @@ -0,0 +1,196 @@ +import { describe, it, expect, beforeEach } from 'bun:test'; +import { registerRules, clearRules, runRules, hasRules } from '../../src/core/rules/engine.js'; +import { rules } from '../../src/core/rules/MissingRenderPartialArguments.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; + +function buildGraph(overrides = {}) { + return buildFactGraph({ + pages: { + 'blog_posts:get': { + path: 'app/views/pages/blog_posts/index.html.liquid', + slug: 'blog_posts', + method: 'get', + renders: ['blog_posts/list'], + render_calls: [{ partial: 'blog_posts/list', args: ['page'] }], + function_calls: [], + }, + }, + partials: { + 'blog_posts/list': { + path: 'app/views/partials/blog_posts/list.liquid', + params: ['page', 'limit'], + renders: ['blog_posts/card'], + render_calls: [{ partial: 'blog_posts/card', args: ['blog_post'] }], + function_calls: [], + rendered_by: ['app/views/pages/blog_posts/index.html.liquid'], + }, + 'blog_posts/card': { + path: 'app/views/partials/blog_posts/card.liquid', + params: ['blog_post'], + renders: [], + render_calls: [], + function_calls: [], + rendered_by: ['app/views/partials/blog_posts/list.liquid'], + }, + 'blog_posts/form': { + path: 'app/views/partials/blog_posts/form.liquid', + params: ['title', 'body', 'errors'], + renders: [], + render_calls: [], + function_calls: [], + rendered_by: [], + }, + }, + commands: {}, + queries: {}, + graphql: {}, + schema: {}, + layouts: {}, + translations: {}, + assets: [], + ...overrides, + }); +} + +describe('MissingRenderPartialArguments rules', () => { + beforeEach(() => { + clearRules(); + registerRules(rules); + }); + + it('registers rules for MissingRenderPartialArguments', () => { + expect(hasRules('MissingRenderPartialArguments')).toBe(true); + }); + + it('shows full signature when target partial has doc params', () => { + const graph = buildGraph(); + const diag = { + check: 'MissingRenderPartialArguments', + params: { partial: 'blog_posts/list', missing_param: 'limit' }, + message: "Missing argument 'limit'", + file: 'app/views/pages/blog_posts/index.html.liquid', + line: 3, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MissingRenderPartialArguments.doc_block_mismatch'); + expect(result.confidence).toBe(0.9); + expect(result.hint_md).toContain('page: page, limit: limit'); + expect(result.hint_md).toContain('`page`'); + expect(result.hint_md).toContain('`limit`'); + expect(result.suggestion).toContain('limit: limit'); + expect(result.see_also).toBeDefined(); + expect(result.see_also.tool).toBe('domain_guide'); + }); + + it('detects chain-satisfied param (caller has param in scope)', () => { + const graph = buildGraph(); + const diag = { + check: 'MissingRenderPartialArguments', + params: { partial: 'blog_posts/card', missing_param: 'page' }, + message: "Missing argument 'page'", + file: 'app/views/partials/blog_posts/list.liquid', + line: 5, + }; + // list.liquid declares 'page' as its own param, so it has it in scope + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + // doc_block_mismatch fires first (priority 10) since card has params + // but chain_satisfied (priority 20) fires when doc_block_mismatch applies + // Actually, doc_block_mismatch wins because card has params ['blog_post'] + // Let me check: for 'page' not in card's params, it still matches doc_block_mismatch + // since card has params.length > 0 + expect(result.rule_id).toBe('MissingRenderPartialArguments.doc_block_mismatch'); + }); + + it('chain_satisfied fires when target has no doc params but caller has the param', () => { + const graph = buildGraph({ + partials: { + 'blog_posts/list': { + path: 'app/views/partials/blog_posts/list.liquid', + params: ['page', 'limit'], + renders: ['blog_posts/undocumented'], + render_calls: [{ partial: 'blog_posts/undocumented', args: [] }], + function_calls: [], + rendered_by: [], + }, + 'blog_posts/undocumented': { + path: 'app/views/partials/blog_posts/undocumented.liquid', + params: [], + renders: [], + render_calls: [], + function_calls: [], + rendered_by: ['app/views/partials/blog_posts/list.liquid'], + }, + }, + }); + const diag = { + check: 'MissingRenderPartialArguments', + params: { partial: 'blog_posts/undocumented', missing_param: 'page' }, + message: "Missing argument 'page'", + file: 'app/views/partials/blog_posts/list.liquid', + line: 5, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + // target has no params → doc_block_mismatch skips → chain_satisfied matches + expect(result.rule_id).toBe('MissingRenderPartialArguments.chain_satisfied'); + expect(result.confidence).toBe(0.85); + expect(result.hint_md).toContain('page'); + expect(result.suggestion).toContain('page: page'); + }); + + it('falls through to generic when no special conditions match', () => { + const graph = buildGraph({ + partials: { + 'blog_posts/orphan': { + path: 'app/views/partials/blog_posts/orphan.liquid', + params: [], + renders: [], + render_calls: [], + function_calls: [], + rendered_by: [], + }, + }, + }); + const diag = { + check: 'MissingRenderPartialArguments', + params: { partial: 'blog_posts/orphan', missing_param: 'x' }, + message: "Missing argument 'x'", + file: 'app/views/pages/blog_posts/index.html.liquid', + line: 1, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MissingRenderPartialArguments.generic'); + expect(result.confidence).toBe(0.5); + }); + + it('handles missing params gracefully', () => { + const graph = buildGraph(); + const diag = { + check: 'MissingRenderPartialArguments', + params: {}, + message: 'some message', + file: 'app/views/pages/blog_posts/index.html.liquid', + line: 1, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MissingRenderPartialArguments.generic'); + }); + + it('includes full param list in signature hint', () => { + const graph = buildGraph(); + const diag = { + check: 'MissingRenderPartialArguments', + params: { partial: 'blog_posts/form', missing_param: 'errors' }, + message: "Missing argument 'errors'", + file: 'app/views/pages/blog_posts/index.html.liquid', + line: 1, + }; + const result = runRules(diag, { graph }); + expect(result.rule_id).toBe('MissingRenderPartialArguments.doc_block_mismatch'); + expect(result.hint_md).toContain('title: title, body: body, errors: errors'); + }); +}); diff --git a/tests/unit/module-scanner-manifest.test.js b/tests/unit/module-scanner-manifest.test.js new file mode 100644 index 0000000..ee2a844 --- /dev/null +++ b/tests/unit/module-scanner-manifest.test.js @@ -0,0 +1,233 @@ +/** + * Module-scanner manifest precedence + drift detection (Phase 4 of the + * pos-cli 6.0.7 alignment plan, 2026-04-25). + * + * Senior-dev contract: `pos-module.json` is the upstream platformOS + * authoritative manifest. `template-values.json` is a generated mirror that + * can drift if module deps are added without re-running `pos-cli modules + * version`. `package.json` is npm metadata — its `version` is unrelated to + * the platformOS module version. The scanner must reflect this hierarchy. + * + * These tests run with throwaway fixtures so they cannot interfere with the + * shared module-scanner.test.js fixture (which only ships template-values.json). + */ + +import { describe, it, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +import { scanModule, listModules } from '../../src/core/module-scanner.js'; + +let projectDir; + +beforeEach(() => { + projectDir = mkdtempSync(join(tmpdir(), 'module-scanner-manifest-')); +}); + +afterEach(() => { + try { rmSync(projectDir, { recursive: true, force: true }); } catch {} +}); + +function write(relPath, content) { + const abs = join(projectDir, relPath); + mkdirSync(join(abs, '..'), { recursive: true }); + writeFileSync(abs, content, 'utf8'); +} + +describe('module-scanner: manifest precedence', () => { + it('pos-module.json wins over template-values.json when both exist', async () => { + write('modules/user/pos-module.json', JSON.stringify({ + machine_name: 'user', + name: 'User', + version: '5.2.8', + dependencies: { core: '^2.1.8', 'common-styling': '^1.11.0', oauth_github: '^0.0.12' }, + })); + write('modules/user/template-values.json', JSON.stringify({ + name: 'User (template-values)', + machine_name: 'user', + type: 'module', + version: '5.2.8', + dependencies: { core: '^2.1.8', 'common-styling': '^1.11.0' }, // missing oauth_github + })); + write('modules/user/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'user'); + + expect(scan.version).toBe('5.2.8'); + // Authoritative dependency list comes from pos-module.json (3 entries). + expect(scan.dependencies.oauth_github).toBe('^0.0.12'); + expect(Object.keys(scan.dependencies)).toHaveLength(3); + expect(scan.manifest_source).toBe('pos-module.json'); + // Display name comes from pos-module.json's `name`, not template-values. + expect(scan.display_name).toBe('User'); + }); + + it('falls back to template-values.json when pos-module.json is absent', async () => { + write('modules/legacy/template-values.json', JSON.stringify({ + name: 'Legacy', + machine_name: 'legacy', + type: 'module', + version: '0.9.0', + dependencies: { core: '^1.0.0' }, + })); + write('modules/legacy/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'legacy'); + + expect(scan.version).toBe('0.9.0'); + expect(scan.dependencies.core).toBe('^1.0.0'); + expect(scan.manifest_source).toBe('template-values.json'); + expect(scan.manifest_warnings).toBeUndefined(); + }); + + it('falls back to package.json when neither platformOS manifest exists', async () => { + write('modules/npm-only/package.json', JSON.stringify({ + name: 'pos-module-npm-only', + version: '3.1.4', + dependencies: { foo: '^1.0.0' }, + })); + write('modules/npm-only/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'npm-only'); + + expect(scan.version).toBe('3.1.4'); + expect(scan.dependencies.foo).toBe('^1.0.0'); + expect(scan.manifest_source).toBe('package.json'); + }); + + it('returns sentinel manifest_source: null when no manifest is present', async () => { + write('modules/bare/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'bare'); + + expect(scan.version).toBe('unknown'); + expect(scan.dependencies).toEqual({}); + expect(scan.manifest_source).toBeNull(); + }); + + it('listModules surfaces manifest_source for every module', async () => { + write('modules/a/pos-module.json', JSON.stringify({ name: 'A', version: '1.0.0', dependencies: {} })); + write('modules/b/template-values.json', JSON.stringify({ name: 'B', version: '2.0.0', dependencies: {} })); + write('modules/c/package.json', JSON.stringify({ name: 'C', version: '3.0.0' })); + + const list = await listModules(projectDir); + const byName = Object.fromEntries(list.map(m => [m.name, m.manifest_source])); + + expect(byName.a).toBe('pos-module.json'); + expect(byName.b).toBe('template-values.json'); + expect(byName.c).toBe('package.json'); + }); + + it('malformed pos-module.json falls through to template-values.json', async () => { + write('modules/broken/pos-module.json', '{ not valid json'); + write('modules/broken/template-values.json', JSON.stringify({ + name: 'Broken', + version: '1.0.0', + dependencies: {}, + })); + write('modules/broken/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'broken'); + + expect(scan.version).toBe('1.0.0'); + expect(scan.manifest_source).toBe('template-values.json'); + }); +}); + +describe('module-scanner: manifest drift detection', () => { + it('flags dependency_drift when pos-module.json adds deps missing from template-values.json', async () => { + write('modules/user/pos-module.json', JSON.stringify({ + name: 'User', + version: '5.2.8', + dependencies: { core: '^2.1.8', 'common-styling': '^1.11.0', oauth_github: '^0.0.12' }, + })); + write('modules/user/template-values.json', JSON.stringify({ + name: 'User', + version: '5.2.8', + dependencies: { core: '^2.1.8', 'common-styling': '^1.11.0' }, + })); + write('modules/user/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'user'); + expect(Array.isArray(scan.manifest_warnings)).toBe(true); + const drift = scan.manifest_warnings.find(w => w.kind === 'dependency_drift'); + expect(drift).toBeDefined(); + expect(drift.only_in_pos_module).toEqual(['oauth_github']); + expect(drift.only_in_template_values).toEqual([]); + expect(drift.message).toMatch(/oauth_github/); + expect(drift.message).toMatch(/pos-cli modules version/); + }); + + it('flags drift in the other direction (template-values has extra deps)', async () => { + write('modules/x/pos-module.json', JSON.stringify({ + name: 'X', + version: '1.0.0', + dependencies: { core: '^1.0.0' }, + })); + write('modules/x/template-values.json', JSON.stringify({ + name: 'X', + version: '1.0.0', + dependencies: { core: '^1.0.0', stray: '^9.9.9' }, + })); + write('modules/x/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'x'); + const drift = scan.manifest_warnings.find(w => w.kind === 'dependency_drift'); + expect(drift.only_in_template_values).toEqual(['stray']); + expect(drift.only_in_pos_module).toEqual([]); + }); + + it('flags version_drift when the two files report different versions', async () => { + write('modules/x/pos-module.json', JSON.stringify({ + name: 'X', + version: '2.0.0', + dependencies: {}, + })); + write('modules/x/template-values.json', JSON.stringify({ + name: 'X', + version: '1.0.0', + dependencies: {}, + })); + write('modules/x/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'x'); + const v = scan.manifest_warnings.find(w => w.kind === 'version_drift'); + expect(v).toBeDefined(); + expect(v.pos_module).toBe('2.0.0'); + expect(v.template_values).toBe('1.0.0'); + // Authoritative version is pos-module.json's. + expect(scan.version).toBe('2.0.0'); + }); + + it('does NOT emit manifest_warnings when both manifests agree', async () => { + write('modules/x/pos-module.json', JSON.stringify({ + name: 'X', + version: '1.0.0', + dependencies: { core: '^1.0.0' }, + })); + write('modules/x/template-values.json', JSON.stringify({ + name: 'X', + version: '1.0.0', + dependencies: { core: '^1.0.0' }, + })); + write('modules/x/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'x'); + expect(scan.manifest_warnings).toBeUndefined(); + }); + + it('does NOT emit manifest_warnings when only one manifest exists', async () => { + // Only pos-module.json — no peer to compare against. + write('modules/a/pos-module.json', JSON.stringify({ + name: 'A', + version: '1.0.0', + dependencies: { core: '^1.0.0' }, + })); + write('modules/a/public/lib/helpers/noop.liquid', '{% return null %}'); + + const scan = await scanModule(projectDir, 'a'); + expect(scan.manifest_warnings).toBeUndefined(); + expect(scan.manifest_source).toBe('pos-module.json'); + }); +}); diff --git a/tests/unit/page-route-index.test.js b/tests/unit/page-route-index.test.js index 3a47ccb..d9e008c 100644 --- a/tests/unit/page-route-index.test.js +++ b/tests/unit/page-route-index.test.js @@ -180,4 +180,91 @@ describe('page-route-index', () => { expect(resolvePageRoute('/totally-unknown', 'get', index)).toEqual({ status: 'missing' }); }); }); + + describe('buildPageRouteIndex with overlay', () => { + it("substitutes the overlay's frontmatter for the on-disk version of the same file", () => { + // Disk version of dashboard.liquid has no method (defaults to GET). + // Overlay declares `method: post` — the route's method set must reflect + // the overlay (POST), not the disk (GET). + const overlay = { + filePath: 'app/views/pages/dashboard.liquid', + content: '---\nmethod: post\n---\n

POST handler

\n', + }; + const { routes } = buildPageRouteIndex(tmpDir, overlay); + const methods = routes.get('dashboard'); + expect(methods).toBeDefined(); + expect(methods.has('post')).toBe(true); + expect(methods.has('get')).toBe(false); + }); + + it('adds a brand-new page (not yet on disk) to the index', () => { + const overlay = { + filePath: 'app/views/pages/contact.liquid', + content: '---\nmethod: post\n---\n

new page

\n', + }; + const { routes } = buildPageRouteIndex(tmpDir, overlay); + expect(routes.has('contact')).toBe(true); + expect(routes.get('contact').has('post')).toBe(true); + }); + + it("respects the overlay's frontmatter slug just like it does for on-disk files", () => { + const overlay = { + filePath: 'app/views/pages/whatever.liquid', + content: '---\nslug: my-custom-route\nmethod: put\n---\n

x

\n', + }; + const { routes } = buildPageRouteIndex(tmpDir, overlay); + expect(routes.has('my-custom-route')).toBe(true); + expect(routes.has('whatever')).toBe(false); + }); + + it('accepts an absolute filePath in the overlay', () => { + const overlay = { + filePath: join(tmpDir, 'app/views/pages/contact.liquid'), + content: '---\nmethod: post\n---\n

x

\n', + }; + const { routes } = buildPageRouteIndex(tmpDir, overlay); + expect(routes.has('contact')).toBe(true); + }); + + it('ignores the overlay when the file is not under app/views/pages/', () => { + // Partials cannot serve routes. The overlay must be silently dropped + // rather than creating a phantom route entry. + const overlay = { + filePath: 'app/views/partials/header.liquid', + content: '---\nslug: phantom\nmethod: post\n---\n

x

\n', + }; + const { routes } = buildPageRouteIndex(tmpDir, overlay); + expect(routes.has('phantom')).toBe(false); + }); + + it('ignores the overlay when filePath does not end in .liquid', () => { + const overlay = { + filePath: 'app/views/pages/contact.json.liquid.bak', + content: '---\nslug: phantom\n---\n', + }; + const { routes } = buildPageRouteIndex(tmpDir, overlay); + expect(routes.has('phantom')).toBe(false); + }); + + it('treats a malformed overlay (missing fields) as if no overlay were provided', () => { + const baseline = buildPageRouteIndex(tmpDir).routes.size; + expect(buildPageRouteIndex(tmpDir, null).routes.size).toBe(baseline); + expect(buildPageRouteIndex(tmpDir, {}).routes.size).toBe(baseline); + expect(buildPageRouteIndex(tmpDir, { filePath: 'x' }).routes.size).toBe(baseline); + expect(buildPageRouteIndex(tmpDir, { content: 'x' }).routes.size).toBe(baseline); + }); + + it('overlay with no frontmatter falls back to path-derived route + GET', () => { + // The disk version of dashboard.liquid is also no-frontmatter / GET. + // Overlay just confirms the same. Ensures empty frontmatter does not + // accidentally erase the file from the index. + const overlay = { + filePath: 'app/views/pages/dashboard.liquid', + content: '

no frontmatter

\n', + }; + const { routes } = buildPageRouteIndex(tmpDir, overlay); + expect(routes.has('dashboard')).toBe(true); + expect(routes.get('dashboard').has('get')).toBe(true); + }); + }); }); diff --git a/tests/unit/probation.test.js b/tests/unit/probation.test.js new file mode 100644 index 0000000..178bb77 --- /dev/null +++ b/tests/unit/probation.test.js @@ -0,0 +1,129 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { openAnalyticsStore } from '../../src/core/analytics-store.js'; +import { resolveProbation } from '../../src/core/case-base.js'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +function tmpPath() { + return join(tmpdir(), `pos-probation-test-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); +} + +function seedDiagnosticsAndOutcomes(store, ruleId, outcomes) { + const windowId = store.insertWindow({ + session_id: 'sess-1', + file: 'test.liquid', + idx: 0, + ts_start: '2026-04-17T10:00:00Z', + ts_end: '2026-04-17T10:01:00Z', + }); + + for (let i = 0; i < outcomes.length; i++) { + const fp = `fp-${ruleId}-${i}`; + store.db.prepare(` + INSERT INTO diagnostics (fp, template_fp, session_id, file, check_name, severity, ts, hint_rule_id, suppressed) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) + `).run(fp, 'tpl-1', 'sess-1', 'test.liquid', 'TestCheck', 'error', '2026-04-17T10:00:00Z', ruleId, 0); + + store.insertOutcome({ + fp, + window_id: windowId, + outcome: outcomes[i], + fix_applied: null, + collateral_added: 0, + }); + } +} + +describe('J5: probation tracking — store operations', () => { + let store; + beforeEach(() => { store = openAnalyticsStore(tmpPath()); }); + afterEach(() => { store.close(); }); + + test('recordPromotion creates a new entry', () => { + store.recordPromotion({ rule_id: 'Test.promoted_1', check_name: 'Test', template_fp: 'tpl1' }); + const promo = store.getPromotion('Test.promoted_1'); + expect(promo).not.toBeNull(); + expect(promo.rule_id).toBe('Test.promoted_1'); + expect(promo.check_name).toBe('Test'); + expect(promo.probation).toBe(1); + expect(promo.resolution).toBeNull(); + }); + + test('getPromotionsOnProbation returns only probation=1', () => { + store.recordPromotion({ rule_id: 'A.rule', check_name: 'A', template_fp: 'tpl-a' }); + store.recordPromotion({ rule_id: 'B.rule', check_name: 'B', template_fp: 'tpl-b' }); + store.resolvePromotion('B.rule', 'kept'); + + const onProbation = store.getPromotionsOnProbation(); + expect(onProbation).toHaveLength(1); + expect(onProbation[0].rule_id).toBe('A.rule'); + }); + + test('resolvePromotion sets probation=0 and resolution', () => { + store.recordPromotion({ rule_id: 'X.rule', check_name: 'X', template_fp: 'tpl-x' }); + store.resolvePromotion('X.rule', 'disabled'); + + const promo = store.getPromotion('X.rule'); + expect(promo.probation).toBe(0); + expect(promo.resolution).toBe('disabled'); + expect(promo.resolved_at).not.toBeNull(); + }); +}); + +describe('J5: probation tracking — resolveProbation', () => { + let store; + beforeEach(() => { store = openAnalyticsStore(tmpPath()); }); + afterEach(() => { store.close(); }); + + test('does not resolve when outcomes < minOutcomes', () => { + store.recordPromotion({ rule_id: 'Test.few', check_name: 'Test', template_fp: 'tpl-1' }); + seedDiagnosticsAndOutcomes(store, 'Test.few', Array(5).fill('resolved')); + + const resolutions = resolveProbation(store, { minOutcomes: 20 }); + expect(resolutions).toHaveLength(0); + + const promo = store.getPromotion('Test.few'); + expect(promo.probation).toBe(1); + }); + + test('keeps rule with high effectiveness', () => { + store.recordPromotion({ rule_id: 'Test.good', check_name: 'Test', template_fp: 'tpl-1' }); + const outcomes = [ + ...Array(15).fill('resolved'), + ...Array(3).fill('unchanged'), + ...Array(2).fill('regressed'), + ]; + seedDiagnosticsAndOutcomes(store, 'Test.good', outcomes); + + const resolutions = resolveProbation(store, { minOutcomes: 20 }); + expect(resolutions).toHaveLength(1); + expect(resolutions[0].rule_id).toBe('Test.good'); + expect(resolutions[0].resolution).toBe('kept'); + expect(resolutions[0].effectiveness).toBeGreaterThan(0.15); + }); + + test('disables rule with low effectiveness', () => { + store.recordPromotion({ rule_id: 'Test.bad', check_name: 'Test', template_fp: 'tpl-1' }); + const outcomes = [ + ...Array(2).fill('resolved'), + ...Array(10).fill('unchanged'), + ...Array(8).fill('regressed'), + ]; + seedDiagnosticsAndOutcomes(store, 'Test.bad', outcomes); + + const resolutions = resolveProbation(store, { minOutcomes: 20 }); + expect(resolutions).toHaveLength(1); + expect(resolutions[0].rule_id).toBe('Test.bad'); + expect(resolutions[0].resolution).toBe('disabled'); + expect(resolutions[0].effectiveness).toBeLessThan(0.15); + }); + + test('skips rules already resolved (not on probation)', () => { + store.recordPromotion({ rule_id: 'Test.done', check_name: 'Test', template_fp: 'tpl-1' }); + store.resolvePromotion('Test.done', 'kept'); + seedDiagnosticsAndOutcomes(store, 'Test.done', Array(25).fill('resolved')); + + const resolutions = resolveProbation(store, { minOutcomes: 20 }); + expect(resolutions).toHaveLength(0); + }); +}); diff --git a/tests/unit/project-fact-graph.test.js b/tests/unit/project-fact-graph.test.js new file mode 100644 index 0000000..69bc546 --- /dev/null +++ b/tests/unit/project-fact-graph.test.js @@ -0,0 +1,232 @@ +import { describe, test, expect } from 'bun:test'; +import { join } from 'node:path'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; +import { scanProject } from '../../src/core/project-scanner.js'; +import { buildDependencyGraph } from '../../src/core/dependency-graph.js'; + +const FIXTURE_DIR = join(import.meta.dir, '..', 'fixtures', 'project'); + +let projectMap; +let graph; + +async function ensureGraph() { + if (!graph) { + projectMap = await scanProject(FIXTURE_DIR); + graph = buildFactGraph(projectMap); + } + return graph; +} + +describe('ProjectFactGraph — construction', () => { + test('builds from fixture project without throwing', async () => { + const g = await ensureGraph(); + expect(g.size).toBeGreaterThan(0); + }); + + test('nodeCount includes all expected types', async () => { + const g = await ensureGraph(); + const counts = g.nodeCount; + expect(counts.page).toBeGreaterThan(0); + expect(counts.partial).toBeGreaterThan(0); + expect(counts.command).toBeGreaterThan(0); + expect(counts.query).toBeGreaterThan(0); + expect(counts.graphql).toBeGreaterThan(0); + expect(counts.schema).toBeGreaterThan(0); + expect(counts.layout).toBeGreaterThan(0); + expect(counts.translation).toBeGreaterThan(0); + expect(counts.asset).toBeGreaterThan(0); + }); + + test('edgeCount is positive', async () => { + const g = await ensureGraph(); + expect(g.edgeCount).toBeGreaterThan(0); + }); +}); + +describe('ProjectFactGraph — node lookups', () => { + test('nodeByPath returns page node', async () => { + const g = await ensureGraph(); + const node = g.nodeByPath('app/views/pages/blog_posts/index.html.liquid'); + expect(node).not.toBeNull(); + expect(node.type).toBe('page'); + expect(node.slug).toBeDefined(); + }); + + test('nodeByPath returns partial node', async () => { + const g = await ensureGraph(); + const node = g.nodeByPath('app/views/partials/blog_posts/card.liquid'); + expect(node).not.toBeNull(); + expect(node.type).toBe('partial'); + }); + + test('nodeByPath returns command node', async () => { + const g = await ensureGraph(); + const cmds = g.nodesByType('command'); + expect(cmds.length).toBeGreaterThan(0); + const cmdPath = cmds[0].path; + expect(g.nodeByPath(cmdPath)).not.toBeNull(); + expect(g.nodeByPath(cmdPath).type).toBe('command'); + }); + + test('nodeByPath returns null for unknown path', async () => { + const g = await ensureGraph(); + expect(g.nodeByPath('app/views/pages/nonexistent.liquid')).toBeNull(); + }); + + test('nodesByType returns all partials', async () => { + const g = await ensureGraph(); + const partials = g.nodesByType('partial'); + const mapPartials = Object.keys(projectMap.partials); + expect(partials.length).toBe(mapPartials.length); + }); + + test('nodeByKey finds partial by name', async () => { + const g = await ensureGraph(); + const node = g.nodeByKey('partial', 'blog_posts/card'); + expect(node).not.toBeNull(); + expect(node.path).toBe('app/views/partials/blog_posts/card.liquid'); + }); + + test('nodeByKey finds graphql by operation path', async () => { + const g = await ensureGraph(); + const node = g.nodeByKey('graphql', 'blog_posts/create'); + expect(node).not.toBeNull(); + expect(node.path).toBe('app/graphql/blog_posts/create.graphql'); + }); + + test('hasNode returns true for existing path', async () => { + const g = await ensureGraph(); + expect(g.hasNode('app/views/partials/blog_posts/card.liquid')).toBe(true); + }); + + test('hasNode returns false for missing path', async () => { + const g = await ensureGraph(); + expect(g.hasNode('app/views/pages/nope.liquid')).toBe(false); + }); +}); + +describe('ProjectFactGraph — edge queries', () => { + test('page depends on rendered partials', async () => { + const g = await ensureGraph(); + const pages = g.nodesByType('page'); + const pageWithRenders = pages.find(p => (p.renders ?? []).length > 0); + if (!pageWithRenders) return; // fixture may not have renders from pages + const deps = g.dependsOn(pageWithRenders.path); + expect(deps.length).toBeGreaterThan(0); + }); + + test('partial is referencedBy its callers', async () => { + const g = await ensureGraph(); + const partials = g.nodesByType('partial'); + const referenced = partials.find(p => g.referencedBy(p.path).length > 0); + expect(referenced).toBeDefined(); + const refs = g.referencedBy(referenced.path); + expect(refs.length).toBeGreaterThan(0); + }); + + test('command depends on graphql operations', async () => { + const g = await ensureGraph(); + const cmds = g.nodesByType('command'); + const cmdWithGql = cmds.find(c => (c.graphql_calls ?? []).length > 0); + if (!cmdWithGql) return; + const deps = g.dependsOn(cmdWithGql.path); + expect(deps.some(d => d.endsWith('.graphql'))).toBe(true); + }); + + test('dependsOn returns empty for leaf node', async () => { + const g = await ensureGraph(); + const assets = g.nodesByType('asset'); + if (assets.length === 0) return; + expect(g.dependsOn(assets[0].path)).toEqual([]); + }); + + test('referencedBy returns empty for unknown path', async () => { + const g = await ensureGraph(); + expect(g.referencedBy('nonexistent')).toEqual([]); + }); +}); + +describe('ProjectFactGraph — file listing', () => { + test('allFiles returns all indexed paths sorted', async () => { + const g = await ensureGraph(); + const files = g.allFiles(); + expect(files.length).toBe(g.size); + for (let i = 1; i < files.length; i++) { + expect(files[i] >= files[i - 1]).toBe(true); + } + }); + + test('allLiquidFiles returns only .liquid files', async () => { + const g = await ensureGraph(); + const liquid = g.allLiquidFiles(); + expect(liquid.length).toBeGreaterThan(0); + expect(liquid.every(f => f.endsWith('.liquid'))).toBe(true); + }); + + test('allCheckableFiles returns .liquid and .graphql', async () => { + const g = await ensureGraph(); + const checkable = g.allCheckableFiles(); + expect(checkable.every(f => f.endsWith('.liquid') || f.endsWith('.graphql'))).toBe(true); + expect(checkable.length).toBeGreaterThanOrEqual(g.allLiquidFiles().length); + }); + + test('allFiles covers all project-map categories', async () => { + const g = await ensureGraph(); + const files = new Set(g.allFiles()); + for (const page of Object.values(projectMap.pages)) { + expect(files.has(page.path)).toBe(true); + } + for (const partial of Object.values(projectMap.partials)) { + expect(files.has(partial.path)).toBe(true); + } + for (const cmdPath of Object.keys(projectMap.commands)) { + expect(files.has(cmdPath)).toBe(true); + } + }); +}); + +describe('ProjectFactGraph — edge integrity invariant', () => { + test('every edge target is a known node or a missing reference', async () => { + const g = await ensureGraph(); + const missing = g.checkEdgeIntegrity(); + for (const { source, target } of missing) { + expect(typeof source).toBe('string'); + expect(typeof target).toBe('string'); + } + }); +}); + +describe('ProjectFactGraph — dependency graph parity', () => { + test('toDependencyGraph matches buildDependencyGraph for static edges', async () => { + const g = await ensureGraph(); + const fromGraph = g.toDependencyGraph(); + const fromLegacy = buildDependencyGraph(projectMap); + + for (const [path, legacyEntry] of Object.entries(fromLegacy)) { + const graphEntry = fromGraph[path]; + if (!graphEntry) continue; + const legacyDeps = new Set(legacyEntry.depends_on); + const graphDeps = new Set(graphEntry.depends_on); + for (const dep of legacyDeps) { + expect(graphDeps.has(dep)).toBe(true); + } + } + }); +}); + +describe('ProjectFactGraph — empty project', () => { + test('handles empty project map gracefully', () => { + const g = buildFactGraph({}); + expect(g.size).toBe(0); + expect(g.edgeCount).toBe(0); + expect(g.allFiles()).toEqual([]); + expect(g.nodeByPath('anything')).toBeNull(); + expect(g.dependsOn('anything')).toEqual([]); + }); + + test('handles partial project map', () => { + const g = buildFactGraph({ pages: {}, partials: {} }); + expect(g.size).toBe(0); + expect(g.nodesByType('page')).toEqual([]); + }); +}); diff --git a/tests/unit/promoted-rules.test.js b/tests/unit/promoted-rules.test.js new file mode 100644 index 0000000..a62bad2 --- /dev/null +++ b/tests/unit/promoted-rules.test.js @@ -0,0 +1,888 @@ +import { describe, it, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdirSync, writeFileSync, rmSync, existsSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { + loadPromotedRules, + readPromotedRulesRaw, + addPromotedRule, + removePromotedRule, + listPromotedRules, +} from '../../src/core/rules/promoted-rules.js'; +import { clearRules, getRulesForCheck, runRules, ruleCount } from '../../src/core/rules/engine.js'; + +let tmpDir; + +function setup() { + tmpDir = join(tmpdir(), `promoted-rules-test-${Date.now()}-${Math.random().toString(36).slice(2)}`); + mkdirSync(join(tmpDir, '.pos-supervisor'), { recursive: true }); +} + +function teardown() { + clearRules(); + try { rmSync(tmpDir, { recursive: true, force: true }); } catch {} +} + +function writeRules(rules) { + writeFileSync( + join(tmpDir, '.pos-supervisor', 'promoted-rules.json'), + JSON.stringify(rules, null, 2), + ); +} + +// ── Loading + compilation ──────────────────────────────────────────────────── + +describe('promoted-rules: loadPromotedRules', () => { + beforeEach(setup); + afterEach(teardown); + + it('returns empty when file does not exist', () => { + rmSync(join(tmpDir, '.pos-supervisor', 'promoted-rules.json'), { force: true }); + const result = loadPromotedRules(tmpDir); + expect(result).toEqual([]); + }); + + it('returns empty for malformed JSON', () => { + writeFileSync(join(tmpDir, '.pos-supervisor', 'promoted-rules.json'), 'not json'); + const result = loadPromotedRules(tmpDir); + expect(result).toEqual([]); + }); + + it('returns empty for non-array JSON', () => { + writeFileSync(join(tmpDir, '.pos-supervisor', 'promoted-rules.json'), '{"a":1}'); + const result = loadPromotedRules(tmpDir); + expect(result).toEqual([]); + }); + + it('loads and compiles a valid rule', () => { + writeRules([{ + id: 'MissingPartial.test_rule', + check: 'MissingPartial', + priority: 55, + when: { param_startsWith: { name: 'modules/' } }, + apply: { hint_md: 'Module partial `{{name}}` not found.', confidence: 0.7 }, + }]); + const compiled = loadPromotedRules(tmpDir); + expect(compiled).toHaveLength(1); + expect(compiled[0].id).toBe('MissingPartial.test_rule'); + expect(compiled[0].check).toBe('MissingPartial'); + expect(compiled[0].priority).toBe(55); + expect(typeof compiled[0].when).toBe('function'); + expect(typeof compiled[0].apply).toBe('function'); + }); + + it('registers compiled rules with the engine', () => { + writeRules([{ + id: 'MissingPartial.promoted_test', + check: 'MissingPartial', + priority: 55, + when: {}, + apply: { hint_md: 'Test hint.', confidence: 0.5 }, + }]); + loadPromotedRules(tmpDir); + const rules = getRulesForCheck('MissingPartial'); + expect(rules.some(r => r.id === 'MissingPartial.promoted_test')).toBe(true); + }); + + it('skips invalid entries without blocking valid ones', () => { + writeRules([ + { id: 'valid_rule', check: 'MissingPartial', apply: { hint_md: 'Works.' } }, + { id: 'bad_rule' }, + { id: 'valid_rule_2', check: 'UnknownFilter', apply: { hint_md: 'Also works.' } }, + ]); + const compiled = loadPromotedRules(tmpDir); + expect(compiled).toHaveLength(2); + expect(compiled.map(r => r.id)).toEqual(['valid_rule', 'valid_rule_2']); + }); + + it('defaults priority to 55 when not specified', () => { + writeRules([{ + id: 'test_default_priority', + check: 'MissingPartial', + apply: { hint_md: 'Test.' }, + }]); + const compiled = loadPromotedRules(tmpDir); + expect(compiled[0].priority).toBe(55); + }); +}); + +// ── Guard predicates (when) ──────────────────────────────────────────────── + +describe('promoted-rules: when guards', () => { + beforeEach(setup); + afterEach(teardown); + + it('param_startsWith matches correctly', () => { + writeRules([{ + id: 'test_startsWith', + check: 'MissingPartial', + when: { param_startsWith: { name: 'modules/' } }, + apply: { hint_md: 'Module partial.', confidence: 0.7 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ params: { name: 'modules/user/form' } })).toBe(true); + expect(guard({ params: { name: 'blog_posts/list' } })).toBe(false); + expect(guard({ params: { name: 123 } })).toBe(false); + expect(guard({ params: {} })).toBe(false); + }); + + it('param_equals matches correctly', () => { + writeRules([{ + id: 'test_equals', + check: 'UnknownFilter', + when: { param_equals: { filter: 'to_json' } }, + apply: { hint_md: 'Use json filter.', confidence: 0.9 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ params: { filter: 'to_json' } })).toBe(true); + expect(guard({ params: { filter: 'downcase' } })).toBe(false); + }); + + it('param_contains matches correctly', () => { + writeRules([{ + id: 'test_contains', + check: 'MissingPartial', + when: { param_contains: { name: 'blog' } }, + apply: { hint_md: 'Blog partial.', confidence: 0.6 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ params: { name: 'blog_posts/form' } })).toBe(true); + expect(guard({ params: { name: 'user/profile' } })).toBe(false); + }); + + it('file_glob matches correctly', () => { + writeRules([{ + id: 'test_glob', + check: 'MissingPartial', + when: { file_glob: 'app/views/partials/**' }, + apply: { hint_md: 'In partials dir.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ file: 'app/views/partials/blog/form.liquid' })).toBe(true); + expect(guard({ file: 'app/views/pages/index.html.liquid' })).toBe(false); + expect(guard({ file: undefined })).toBe(false); + }); + + it('file_type matches correctly', () => { + writeRules([{ + id: 'test_filetype', + check: 'MissingPartial', + when: { file_type: 'page' }, + apply: { hint_md: 'In a page.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ file: 'app/views/pages/index.html.liquid' })).toBe(true); + expect(guard({ file: 'app/views/partials/card.liquid' })).toBe(false); + expect(guard({ file: null })).toBe(false); + }); + + it('multiple guards must all match (AND)', () => { + writeRules([{ + id: 'test_multi_guard', + check: 'MissingPartial', + when: { + param_startsWith: { name: 'modules/' }, + file_type: 'page', + }, + apply: { hint_md: 'Module partial in page.', confidence: 0.8 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ params: { name: 'modules/user/form' }, file: 'app/views/pages/index.html.liquid' })).toBe(true); + expect(guard({ params: { name: 'modules/user/form' }, file: 'app/views/partials/x.liquid' })).toBe(false); + expect(guard({ params: { name: 'blog/form' }, file: 'app/views/pages/index.html.liquid' })).toBe(false); + }); + + it('empty when matches everything', () => { + writeRules([{ + id: 'test_empty_when', + check: 'MissingPartial', + when: {}, + apply: { hint_md: 'Generic.', confidence: 0.3 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ params: {}, file: 'anything.liquid' })).toBe(true); + expect(guard({})).toBe(true); + }); +}); + +// ── Apply (hint generation) ────────────────────────────────────────────── + +describe('promoted-rules: apply', () => { + beforeEach(setup); + afterEach(teardown); + + it('interpolates {{param}} in hint_md', () => { + writeRules([{ + id: 'test_interpolate', + check: 'MissingPartial', + when: {}, + apply: { hint_md: 'Partial `{{name}}` not found. Check `{{path}}`.', confidence: 0.7 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const result = compiled[0].apply({ params: { name: 'blog/form', path: 'app/views/partials' } }); + + expect(result.rule_id).toBe('test_interpolate'); + expect(result.hint_md).toBe('Partial `blog/form` not found. Check `app/views/partials`.'); + expect(result.confidence).toBe(0.7); + }); + + it('preserves unmatched template vars', () => { + writeRules([{ + id: 'test_unmatched', + check: 'MissingPartial', + when: {}, + apply: { hint_md: 'Missing `{{name}}` at `{{location}}`.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const result = compiled[0].apply({ params: { name: 'form' } }); + + expect(result.hint_md).toBe('Missing `form` at `{{location}}`.'); + }); + + it('includes see_also when specified', () => { + writeRules([{ + id: 'test_see_also', + check: 'MissingPartial', + when: {}, + apply: { + hint_md: 'Check modules.', + confidence: 0.6, + see_also: { tool: 'module_info', args: { aspect: 'api' } }, + }, + }]); + const compiled = loadPromotedRules(tmpDir); + const result = compiled[0].apply({ params: {} }); + + expect(result.see_also).toEqual({ tool: 'module_info', args: { aspect: 'api' } }); + }); + + it('defaults confidence to 0.5 when not specified', () => { + writeRules([{ + id: 'test_default_confidence', + check: 'MissingPartial', + when: {}, + apply: { hint_md: 'Test.' }, + }]); + const compiled = loadPromotedRules(tmpDir); + const result = compiled[0].apply({ params: {} }); + + expect(result.confidence).toBe(0.5); + }); +}); + +// ── Engine integration ──────────────────────────────────────────────────── + +describe('promoted-rules: engine integration', () => { + beforeEach(setup); + afterEach(teardown); + + it('promoted rule fires via runRules', () => { + writeRules([{ + id: 'MissingPartial.promoted_module', + check: 'MissingPartial', + priority: 55, + when: { param_startsWith: { name: 'modules/' } }, + apply: { hint_md: 'Module partial `{{name}}` not found.', confidence: 0.7 }, + }]); + loadPromotedRules(tmpDir); + + const diag = { check: 'MissingPartial', params: { name: 'modules/user/form' } }; + const result = runRules(diag, {}); + + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MissingPartial.promoted_module'); + expect(result.hint_md).toBe('Module partial `modules/user/form` not found.'); + }); + + it('promoted rule does not fire when guard fails', () => { + writeRules([{ + id: 'MissingPartial.promoted_module_only', + check: 'MissingPartial', + priority: 55, + when: { param_startsWith: { name: 'modules/' } }, + apply: { hint_md: 'Module partial.', confidence: 0.7 }, + }]); + loadPromotedRules(tmpDir); + + const diag = { check: 'MissingPartial', params: { name: 'blog_posts/list' } }; + const result = runRules(diag, {}); + + expect(result).toBeNull(); + }); +}); + +// ── CRUD operations ────────────────────────────────────────────────────── + +describe('promoted-rules: CRUD', () => { + beforeEach(setup); + afterEach(teardown); + + it('addPromotedRule creates the file and persists the rule', () => { + const entry = { + id: 'Test.new_rule', + check: 'Test', + apply: { hint_md: 'New rule hint.' }, + }; + addPromotedRule(tmpDir, entry); + + const raw = readPromotedRulesRaw(tmpDir); + expect(raw).toHaveLength(1); + expect(raw[0].id).toBe('Test.new_rule'); + }); + + it('addPromotedRule rejects duplicate IDs', () => { + const entry = { id: 'Test.dup', check: 'Test', apply: { hint_md: 'Dup.' } }; + addPromotedRule(tmpDir, entry); + + expect(() => addPromotedRule(tmpDir, entry)).toThrow('already exists'); + }); + + it('addPromotedRule validates the entry before persisting', () => { + expect(() => addPromotedRule(tmpDir, { id: 'bad' })).toThrow('missing required fields'); + }); + + it('removePromotedRule removes an existing rule', () => { + addPromotedRule(tmpDir, { id: 'Test.removeme', check: 'Test', apply: { hint_md: 'Remove me.' } }); + removePromotedRule(tmpDir, 'Test.removeme'); + + const raw = readPromotedRulesRaw(tmpDir); + expect(raw).toHaveLength(0); + }); + + it('removePromotedRule throws for unknown rule', () => { + expect(() => removePromotedRule(tmpDir, 'Test.nonexistent')).toThrow('not found'); + }); + + it('listPromotedRules returns all raw rules', () => { + addPromotedRule(tmpDir, { id: 'A.rule1', check: 'A', apply: { hint_md: 'One.' } }); + addPromotedRule(tmpDir, { id: 'B.rule2', check: 'B', apply: { hint_md: 'Two.' } }); + + const list = listPromotedRules(tmpDir); + expect(list).toHaveLength(2); + expect(list.map(r => r.id)).toEqual(['A.rule1', 'B.rule2']); + }); + + it('listPromotedRules returns empty when no file exists', () => { + rmSync(join(tmpDir, '.pos-supervisor', 'promoted-rules.json'), { force: true }); + expect(listPromotedRules(tmpDir)).toEqual([]); + }); +}); + +// ── Internal helpers ─────────────────────────────────────────────────────── + +describe('promoted-rules: file_type classification', () => { + beforeEach(setup); + afterEach(teardown); + + const testCases = [ + ['app/views/pages/index.html.liquid', 'page'], + ['app/views/partials/card.liquid', 'partial'], + ['app/views/layouts/main.liquid', 'layout'], + ['app/lib/commands/blog/create.liquid', 'command'], + ['app/lib/queries/blog/search.liquid', 'query'], + ['app/graphql/blog/create.graphql', 'graphql'], + ['app/schema/blog.yml', 'schema'], + ['modules/user/form.liquid', 'module'], + ]; + + for (const [path, expectedType] of testCases) { + it(`classifies ${path} as ${expectedType}`, () => { + writeRules([{ + id: `test_classify_${expectedType}`, + check: 'TestCheck', + when: { file_type: expectedType }, + apply: { hint_md: 'Match.' }, + }]); + const compiled = loadPromotedRules(tmpDir); + expect(compiled[0].when({ file: path })).toBe(true); + }); + } +}); + +describe('promoted-rules: glob patterns', () => { + beforeEach(setup); + afterEach(teardown); + + it('** matches nested directories', () => { + writeRules([{ + id: 'test_globstar', + check: 'Test', + when: { file_glob: 'app/views/**/*.liquid' }, + apply: { hint_md: 'Match.' }, + }]); + const compiled = loadPromotedRules(tmpDir); + expect(compiled[0].when({ file: 'app/views/pages/blog/index.html.liquid' })).toBe(true); + expect(compiled[0].when({ file: 'app/lib/commands/x.liquid' })).toBe(false); + }); + + it('* matches single path segment', () => { + writeRules([{ + id: 'test_star', + check: 'Test', + when: { file_glob: 'app/views/pages/*.liquid' }, + apply: { hint_md: 'Match.' }, + }]); + const compiled = loadPromotedRules(tmpDir); + expect(compiled[0].when({ file: 'app/views/pages/index.liquid' })).toBe(true); + expect(compiled[0].when({ file: 'app/views/pages/blog/index.liquid' })).toBe(false); + }); +}); + +// ── Graph-aware guards ────────────────────────────────────────────────────── + +function mockGraph(nodes = {}, edges = {}) { + return { + referencedBy(filePath) { + return edges[filePath] ?? []; + }, + hasNode(filePath) { + return filePath in nodes; + }, + nodeByPath(filePath) { + return nodes[filePath] ?? null; + }, + }; +} + +describe('promoted-rules: graph-aware guards', () => { + beforeEach(setup); + afterEach(teardown); + + it('has_callers: true matches when file has callers', () => { + writeRules([{ + id: 'test_has_callers_true', + check: 'TestCheck', + when: { has_callers: true }, + apply: { hint_md: 'Has callers.', confidence: 0.6 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/card.liquid': { type: 'partial', key: 'card' } }, + { 'app/views/partials/card.liquid': ['app/views/pages/index.liquid'] }, + ); + + expect(guard({ file: 'app/views/partials/card.liquid' }, { graph })).toBe(true); + }); + + it('has_callers: true rejects when file has no callers', () => { + writeRules([{ + id: 'test_has_callers_true_no_refs', + check: 'TestCheck', + when: { has_callers: true }, + apply: { hint_md: 'Has callers.', confidence: 0.6 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/orphan.liquid': { type: 'partial', key: 'orphan' } }, + { 'app/views/partials/orphan.liquid': [] }, + ); + + expect(guard({ file: 'app/views/partials/orphan.liquid' }, { graph })).toBe(false); + }); + + it('has_callers: false matches when file has no callers', () => { + writeRules([{ + id: 'test_has_callers_false', + check: 'TestCheck', + when: { has_callers: false }, + apply: { hint_md: 'No callers.', confidence: 0.6 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/orphan.liquid': { type: 'partial', key: 'orphan' } }, + { 'app/views/partials/orphan.liquid': [] }, + ); + + expect(guard({ file: 'app/views/partials/orphan.liquid' }, { graph })).toBe(true); + }); + + it('caller_count_gte matches when callers >= threshold', () => { + writeRules([{ + id: 'test_caller_count', + check: 'TestCheck', + when: { caller_count_gte: 3 }, + apply: { hint_md: 'Many callers.', confidence: 0.7 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/header.liquid': { type: 'partial', key: 'header' } }, + { 'app/views/partials/header.liquid': ['page1.liquid', 'page2.liquid', 'page3.liquid'] }, + ); + + expect(guard({ file: 'app/views/partials/header.liquid' }, { graph })).toBe(true); + }); + + it('caller_count_gte rejects when callers < threshold', () => { + writeRules([{ + id: 'test_caller_count_below', + check: 'TestCheck', + when: { caller_count_gte: 3 }, + apply: { hint_md: 'Many callers.', confidence: 0.7 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/header.liquid': { type: 'partial', key: 'header' } }, + { 'app/views/partials/header.liquid': ['page1.liquid', 'page2.liquid'] }, + ); + + expect(guard({ file: 'app/views/partials/header.liquid' }, { graph })).toBe(false); + }); + + it('has_params: true matches when file has doc params', () => { + writeRules([{ + id: 'test_has_params_true', + check: 'TestCheck', + when: { has_params: true }, + apply: { hint_md: 'Documented partial.', confidence: 0.6 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph({ + 'app/views/partials/card.liquid': { type: 'partial', key: 'card', params: ['title', 'image'] }, + }); + + expect(guard({ file: 'app/views/partials/card.liquid' }, { graph })).toBe(true); + }); + + it('has_params: false matches when file has no doc params', () => { + writeRules([{ + id: 'test_has_params_false', + check: 'TestCheck', + when: { has_params: false }, + apply: { hint_md: 'Undocumented partial.', confidence: 0.6 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph({ + 'app/views/partials/card.liquid': { type: 'partial', key: 'card', params: [] }, + }); + + expect(guard({ file: 'app/views/partials/card.liquid' }, { graph })).toBe(true); + }); + + it('has_params: true rejects when file has no params', () => { + writeRules([{ + id: 'test_has_params_true_no_params', + check: 'TestCheck', + when: { has_params: true }, + apply: { hint_md: 'Documented.', confidence: 0.6 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph({ + 'app/views/partials/card.liquid': { type: 'partial', key: 'card', params: [] }, + }); + + expect(guard({ file: 'app/views/partials/card.liquid' }, { graph })).toBe(false); + }); + + it('is_orphan: true matches orphan files', () => { + writeRules([{ + id: 'test_is_orphan_true', + check: 'TestCheck', + when: { is_orphan: true }, + apply: { hint_md: 'Orphan file.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/dead.liquid': { type: 'partial', key: 'dead' } }, + { 'app/views/partials/dead.liquid': [] }, + ); + + expect(guard({ file: 'app/views/partials/dead.liquid' }, { graph })).toBe(true); + }); + + it('is_orphan: false matches non-orphan files', () => { + writeRules([{ + id: 'test_is_orphan_false', + check: 'TestCheck', + when: { is_orphan: false }, + apply: { hint_md: 'Not orphan.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/used.liquid': { type: 'partial', key: 'used' } }, + { 'app/views/partials/used.liquid': ['page.liquid'] }, + ); + + expect(guard({ file: 'app/views/partials/used.liquid' }, { graph })).toBe(true); + }); + + it('is_orphan: true rejects when file has callers', () => { + writeRules([{ + id: 'test_is_orphan_true_has_callers', + check: 'TestCheck', + when: { is_orphan: true }, + apply: { hint_md: 'Orphan.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/used.liquid': { type: 'partial', key: 'used' } }, + { 'app/views/partials/used.liquid': ['caller.liquid'] }, + ); + + expect(guard({ file: 'app/views/partials/used.liquid' }, { graph })).toBe(false); + }); + + it('graph guards degrade gracefully when no graph provided', () => { + writeRules([{ + id: 'test_no_graph', + check: 'TestCheck', + when: { has_callers: true }, + apply: { hint_md: 'Needs callers.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + + expect(guard({ file: 'app/views/partials/card.liquid' }, {})).toBe(false); + expect(guard({ file: 'app/views/partials/card.liquid' }, null)).toBe(false); + expect(guard({ file: 'app/views/partials/card.liquid' })).toBe(false); + }); + + it('graph guards degrade gracefully when no diag.file', () => { + writeRules([{ + id: 'test_no_file', + check: 'TestCheck', + when: { is_orphan: true }, + apply: { hint_md: 'Orphan.', confidence: 0.5 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph({ 'app/views/partials/x.liquid': { type: 'partial' } }); + + expect(guard({}, { graph })).toBe(false); + expect(guard({ file: null }, { graph })).toBe(false); + expect(guard({ file: undefined }, { graph })).toBe(false); + }); + + it('combines graph guards with param guards (AND semantics)', () => { + writeRules([{ + id: 'test_combined_graph_param', + check: 'MissingPartial', + when: { + param_startsWith: { name: 'modules/' }, + has_callers: true, + is_orphan: false, + }, + apply: { hint_md: 'Module partial with callers.', confidence: 0.8 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/mod.liquid': { type: 'partial', key: 'mod' } }, + { 'app/views/partials/mod.liquid': ['page.liquid'] }, + ); + + expect(guard( + { params: { name: 'modules/user/form' }, file: 'app/views/partials/mod.liquid' }, + { graph }, + )).toBe(true); + + expect(guard( + { params: { name: 'blog/form' }, file: 'app/views/partials/mod.liquid' }, + { graph }, + )).toBe(false); + + expect(guard( + { params: { name: 'modules/user/form' }, file: 'app/views/partials/mod.liquid' }, + {}, + )).toBe(false); + }); + + it('combines file_type + graph guards', () => { + writeRules([{ + id: 'test_filetype_graph', + check: 'TestCheck', + when: { + file_type: 'partial', + caller_count_gte: 2, + has_params: true, + }, + apply: { hint_md: 'Popular documented partial.', confidence: 0.9 }, + }]); + const compiled = loadPromotedRules(tmpDir); + const guard = compiled[0].when; + const graph = mockGraph( + { 'app/views/partials/card.liquid': { type: 'partial', key: 'card', params: ['title'] } }, + { 'app/views/partials/card.liquid': ['page1.liquid', 'page2.liquid'] }, + ); + + expect(guard( + { file: 'app/views/partials/card.liquid' }, + { graph }, + )).toBe(true); + + const graphFewCallers = mockGraph( + { 'app/views/partials/card.liquid': { type: 'partial', key: 'card', params: ['title'] } }, + { 'app/views/partials/card.liquid': ['page1.liquid'] }, + ); + expect(guard( + { file: 'app/views/partials/card.liquid' }, + { graph: graphFewCallers }, + )).toBe(false); + }); + + it('engine integration: graph-aware rule fires via runRules', () => { + writeRules([{ + id: 'TestCheck.graph_rule', + check: 'TestCheck', + priority: 55, + when: { has_callers: true, caller_count_gte: 1 }, + apply: { hint_md: 'File has callers.', confidence: 0.7 }, + }]); + loadPromotedRules(tmpDir); + + const diag = { check: 'TestCheck', file: 'app/views/partials/card.liquid' }; + const graph = mockGraph( + { 'app/views/partials/card.liquid': { type: 'partial', key: 'card' } }, + { 'app/views/partials/card.liquid': ['page.liquid'] }, + ); + const result = runRules(diag, { graph }); + + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('TestCheck.graph_rule'); + }); + + it('engine integration: graph-aware rule skipped when guard fails', () => { + writeRules([{ + id: 'TestCheck.graph_only_callers', + check: 'TestCheck', + priority: 55, + when: { caller_count_gte: 5 }, + apply: { hint_md: 'Very popular.', confidence: 0.9 }, + }]); + loadPromotedRules(tmpDir); + + const diag = { check: 'TestCheck', file: 'app/views/partials/card.liquid' }; + const graph = mockGraph( + { 'app/views/partials/card.liquid': { type: 'partial', key: 'card' } }, + { 'app/views/partials/card.liquid': ['page.liquid', 'page2.liquid'] }, + ); + const result = runRules(diag, { graph }); + + expect(result).toBeNull(); + }); +}); + +// ── Query helpers ─────────────────────────────────────────────────────────── + +import { callerCount, isOrphan, hasDocParams, classifyFileType } from '../../src/core/rules/queries.js'; + +describe('queries: callerCount', () => { + it('returns caller count from graph', () => { + const graph = mockGraph({}, { 'a.liquid': ['b.liquid', 'c.liquid'] }); + expect(callerCount(graph, 'a.liquid')).toBe(2); + }); + + it('returns 0 for file with no callers', () => { + const graph = mockGraph({}, { 'a.liquid': [] }); + expect(callerCount(graph, 'a.liquid')).toBe(0); + }); + + it('returns 0 when graph is null', () => { + expect(callerCount(null, 'a.liquid')).toBe(0); + }); + + it('returns 0 when filePath is null', () => { + const graph = mockGraph({}, {}); + expect(callerCount(graph, null)).toBe(0); + }); +}); + +describe('queries: isOrphan', () => { + it('returns true for file in graph with no callers', () => { + const graph = mockGraph( + { 'a.liquid': { type: 'partial' } }, + { 'a.liquid': [] }, + ); + expect(isOrphan(graph, 'a.liquid')).toBe(true); + }); + + it('returns false for file with callers', () => { + const graph = mockGraph( + { 'a.liquid': { type: 'partial' } }, + { 'a.liquid': ['b.liquid'] }, + ); + expect(isOrphan(graph, 'a.liquid')).toBe(false); + }); + + it('returns false for file not in graph', () => { + const graph = mockGraph({}, {}); + expect(isOrphan(graph, 'unknown.liquid')).toBe(false); + }); + + it('returns false when graph is null', () => { + expect(isOrphan(null, 'a.liquid')).toBe(false); + }); +}); + +describe('queries: hasDocParams', () => { + it('returns true when node has params array with entries', () => { + const graph = mockGraph({ 'a.liquid': { params: ['title', 'image'] } }); + expect(hasDocParams(graph, 'a.liquid')).toBe(true); + }); + + it('returns false when node has empty params array', () => { + const graph = mockGraph({ 'a.liquid': { params: [] } }); + expect(hasDocParams(graph, 'a.liquid')).toBe(false); + }); + + it('returns false when node has no params field', () => { + const graph = mockGraph({ 'a.liquid': { type: 'partial' } }); + expect(hasDocParams(graph, 'a.liquid')).toBe(false); + }); + + it('returns false when node not found', () => { + const graph = mockGraph({}); + expect(hasDocParams(graph, 'unknown.liquid')).toBe(false); + }); + + it('returns false when graph is null', () => { + expect(hasDocParams(null, 'a.liquid')).toBe(false); + }); +}); + +describe('queries: classifyFileType', () => { + const cases = [ + ['app/views/pages/index.html.liquid', 'page'], + ['app/views/partials/card.liquid', 'partial'], + ['app/views/layouts/main.liquid', 'layout'], + ['app/lib/commands/blog/create.liquid', 'command'], + ['app/lib/queries/blog/search.liquid', 'query'], + ['app/graphql/blog/create.graphql', 'graphql'], + ['app/schema/blog.yml', 'schema'], + ['modules/user/form.liquid', 'module'], + ['some/other/path.liquid', 'unknown'], + [null, 'unknown'], + [undefined, 'unknown'], + ['', 'unknown'], + ]; + + for (const [input, expected] of cases) { + it(`classifies ${JSON.stringify(input)} as ${expected}`, () => { + expect(classifyFileType(input)).toBe(expected); + }); + } +}); diff --git a/tests/unit/render-flow.test.js b/tests/unit/render-flow.test.js new file mode 100644 index 0000000..1c757e3 --- /dev/null +++ b/tests/unit/render-flow.test.js @@ -0,0 +1,299 @@ +import { describe, it, expect } from 'bun:test'; +import { extractAll } from '../../src/core/liquid-parser.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; +import { + isVariablePassedToRender, + isVariablePassedToFunction, + callersWithArgs, + getPartialParams, + missingArgsForCaller, + isParamAvailableInCallerScope, + renderFlowSummary, +} from '../../src/core/render-flow.js'; + +// ── liquid-parser: renderCalls extraction ─────────────────────────────────── + +describe('liquid-parser renderCalls extraction', () => { + it('extracts named args from structured render markup', () => { + const result = extractAll(`{% render 'form', title: title, body: body %}`); + expect(result.renderCalls).toHaveLength(1); + expect(result.renderCalls[0].partial).toBe('form'); + expect(result.renderCalls[0].args).toEqual(['title', 'body']); + }); + + it('extracts named args from render with no args', () => { + const result = extractAll(`{% render 'simple' %}`); + expect(result.renderCalls).toHaveLength(1); + expect(result.renderCalls[0].partial).toBe('simple'); + expect(result.renderCalls[0].args).toEqual([]); + }); + + it('handles multiple render calls in one file', () => { + const result = extractAll(` + {% render 'header', theme: theme %} + {% render 'footer', year: year %} + `); + expect(result.renderCalls).toHaveLength(2); + expect(result.renderCalls[0]).toEqual({ partial: 'header', args: ['theme'] }); + expect(result.renderCalls[1]).toEqual({ partial: 'footer', args: ['year'] }); + }); + + it('extracts args from include tags', () => { + const result = extractAll(`{% include 'legacy', mode: mode %}`); + expect(result.renderCalls).toHaveLength(1); + expect(result.renderCalls[0].partial).toBe('legacy'); + expect(result.renderCalls[0].args).toEqual(['mode']); + }); + + it('preserves backward-compatible renders array', () => { + const result = extractAll(` + {% render 'a', x: x %} + {% render 'b' %} + `); + expect(result.renders).toEqual(['a', 'b']); + expect(result.renderCalls).toHaveLength(2); + }); + + it('handles render with path-style partial names', () => { + const result = extractAll(`{% render 'blog_posts/card', blog_post: item %}`); + expect(result.renderCalls).toHaveLength(1); + expect(result.renderCalls[0].partial).toBe('blog_posts/card'); + expect(result.renderCalls[0].args).toEqual(['blog_post']); + }); + + it('returns empty renderCalls for unparseable content', () => { + const result = extractAll('{% invalid unclosed'); + // extractAll returns null for completely unparseable content + // or an object with empty renderCalls for partial failures + if (result) { + expect(result.renderCalls).toBeDefined(); + } + }); +}); + +// ── ProjectFactGraph: render call indexing ────────────────────────────────── + +function buildTestGraph(overrides = {}) { + const base = { + pages: { + 'blog_posts:get': { + path: 'app/views/pages/blog_posts/index.html.liquid', + slug: 'blog_posts', + method: 'get', + renders: ['blog_posts/list'], + render_calls: [{ partial: 'blog_posts/list', args: ['page'] }], + function_calls: [], + }, + }, + partials: { + 'blog_posts/list': { + path: 'app/views/partials/blog_posts/list.liquid', + params: ['page', 'limit'], + renders: ['blog_posts/card'], + render_calls: [{ partial: 'blog_posts/card', args: ['blog_post'] }], + function_calls: [{ variable: 'items', path: 'queries/blog_posts/search' }], + rendered_by: ['app/views/pages/blog_posts/index.html.liquid'], + }, + 'blog_posts/card': { + path: 'app/views/partials/blog_posts/card.liquid', + params: ['blog_post'], + renders: [], + render_calls: [], + function_calls: [], + rendered_by: ['app/views/partials/blog_posts/list.liquid'], + }, + 'blog_posts/form': { + path: 'app/views/partials/blog_posts/form.liquid', + params: ['title', 'body', 'errors'], + renders: [], + render_calls: [], + function_calls: [], + rendered_by: [], + }, + }, + commands: {}, + queries: { + 'app/lib/queries/blog_posts/search.liquid': { + params: ['query'], + graphql_calls: [], + function_calls: [], + }, + }, + graphql: {}, + schema: {}, + layouts: {}, + translations: {}, + assets: [], + ...overrides, + }; + return buildFactGraph(base); +} + +describe('ProjectFactGraph render call queries', () => { + it('renderCallsFrom returns calls for a file', () => { + const graph = buildTestGraph(); + const calls = graph.renderCallsFrom('app/views/partials/blog_posts/list.liquid'); + expect(calls).toHaveLength(1); + expect(calls[0].partial).toBe('blog_posts/card'); + expect(calls[0].args).toEqual(['blog_post']); + }); + + it('renderCallsFrom returns empty for file with no render calls', () => { + const graph = buildTestGraph(); + const calls = graph.renderCallsFrom('app/views/partials/blog_posts/card.liquid'); + expect(calls).toEqual([]); + }); + + it('renderCallsTo returns all callers of a partial', () => { + const graph = buildTestGraph(); + const callers = graph.renderCallsTo('blog_posts/list'); + expect(callers).toHaveLength(1); + expect(callers[0].callerPath).toBe('app/views/pages/blog_posts/index.html.liquid'); + expect(callers[0].args).toEqual(['page']); + }); + + it('partialSignature returns declared params', () => { + const graph = buildTestGraph(); + expect(graph.partialSignature('blog_posts/list')).toEqual(['page', 'limit']); + expect(graph.partialSignature('blog_posts/card')).toEqual(['blog_post']); + }); + + it('partialSignature returns null for unknown partial', () => { + const graph = buildTestGraph(); + expect(graph.partialSignature('nonexistent')).toBeNull(); + }); +}); + +// ── render-flow.js: pure query functions ──────────────────────────────────── + +describe('isVariablePassedToRender', () => { + it('returns true when variable matches a render arg name', () => { + const graph = buildTestGraph(); + expect(isVariablePassedToRender(graph, 'app/views/partials/blog_posts/list.liquid', 'blog_post')).toBe(true); + }); + + it('returns false when variable is not in any render arg', () => { + const graph = buildTestGraph(); + expect(isVariablePassedToRender(graph, 'app/views/partials/blog_posts/list.liquid', 'unrelated')).toBe(false); + }); + + it('returns false for file with no render calls', () => { + const graph = buildTestGraph(); + expect(isVariablePassedToRender(graph, 'app/views/partials/blog_posts/card.liquid', 'blog_post')).toBe(false); + }); +}); + +describe('isVariablePassedToFunction', () => { + it('returns true when variable matches a function call result variable', () => { + const graph = buildTestGraph(); + expect(isVariablePassedToFunction(graph, 'app/views/partials/blog_posts/list.liquid', 'items')).toBe(true); + }); + + it('returns false for non-matching variable', () => { + const graph = buildTestGraph(); + expect(isVariablePassedToFunction(graph, 'app/views/partials/blog_posts/list.liquid', 'other')).toBe(false); + }); +}); + +describe('callersWithArgs', () => { + it('returns callers with their passed arguments', () => { + const graph = buildTestGraph(); + const callers = callersWithArgs(graph, 'blog_posts/card'); + expect(callers).toHaveLength(1); + expect(callers[0].callerPath).toBe('app/views/partials/blog_posts/list.liquid'); + expect(callers[0].args).toEqual(['blog_post']); + }); + + it('returns empty for partial with no callers', () => { + const graph = buildTestGraph(); + const callers = callersWithArgs(graph, 'blog_posts/form'); + expect(callers).toEqual([]); + }); +}); + +describe('missingArgsForCaller', () => { + it('detects when caller passes all required params', () => { + const graph = buildTestGraph(); + const missing = missingArgsForCaller( + graph, + 'app/views/partials/blog_posts/list.liquid', + 'blog_posts/card', + ); + expect(missing).toEqual([]); + }); + + it('detects missing params when caller omits some', () => { + const graph = buildTestGraph(); + const missing = missingArgsForCaller( + graph, + 'app/views/pages/blog_posts/index.html.liquid', + 'blog_posts/list', + ); + // Page passes 'page' but not 'limit' + expect(missing).toEqual(['limit']); + }); + + it('returns all params when caller has no render call to target', () => { + const graph = buildTestGraph(); + const missing = missingArgsForCaller( + graph, + 'app/views/partials/blog_posts/card.liquid', + 'blog_posts/form', + ); + expect(missing).toEqual(['title', 'body', 'errors']); + }); +}); + +describe('isParamAvailableInCallerScope', () => { + it('returns true when caller declares the param', () => { + const graph = buildTestGraph(); + expect(isParamAvailableInCallerScope( + graph, + 'app/views/partials/blog_posts/list.liquid', + 'page', + )).toBe(true); + }); + + it('returns false when caller does not declare the param', () => { + const graph = buildTestGraph(); + expect(isParamAvailableInCallerScope( + graph, + 'app/views/partials/blog_posts/list.liquid', + 'unrelated', + )).toBe(false); + }); + + it('returns false for pages (no params)', () => { + const graph = buildTestGraph(); + expect(isParamAvailableInCallerScope( + graph, + 'app/views/pages/blog_posts/index.html.liquid', + 'page', + )).toBe(false); + }); +}); + +describe('renderFlowSummary', () => { + it('shows passed args, declared params, and missing args', () => { + const graph = buildTestGraph(); + const summary = renderFlowSummary(graph, 'app/views/pages/blog_posts/index.html.liquid'); + expect(summary).toHaveLength(1); + expect(summary[0].partial).toBe('blog_posts/list'); + expect(summary[0].passed_args).toEqual(['page']); + expect(summary[0].declared_params).toEqual(['page', 'limit']); + expect(summary[0].missing_args).toEqual(['limit']); + }); + + it('shows no missing args when all are passed', () => { + const graph = buildTestGraph(); + const summary = renderFlowSummary(graph, 'app/views/partials/blog_posts/list.liquid'); + expect(summary).toHaveLength(1); + expect(summary[0].missing_args).toEqual([]); + }); + + it('returns empty for file with no render calls', () => { + const graph = buildTestGraph(); + const summary = renderFlowSummary(graph, 'app/views/partials/blog_posts/card.liquid'); + expect(summary).toEqual([]); + }); +}); diff --git a/tests/unit/rule-engine-overrides.test.js b/tests/unit/rule-engine-overrides.test.js new file mode 100644 index 0000000..2ab5755 --- /dev/null +++ b/tests/unit/rule-engine-overrides.test.js @@ -0,0 +1,92 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { + registerRule, clearRules, runRules, + updateDisabledRules, updateForceOverrides, + setDisabledRuleDetails, getDisabledRuleDetails, + isCheckForceDisabled, +} from '../../src/core/rules/engine.js'; + +function makeRule(id, check, response) { + return { + id, check, priority: 10, + when: () => true, + apply: () => ({ rule_id: id, hint_md: response, fixes: [], confidence: 0.8 }), + }; +} + +// Reset module-level engine state so tests don't leak into other files +// (the engine registry + override sets are singletons; any assertion here +// that mutates them must tidy up, otherwise the next file runs with +// `_forceDisabled` still populated and legitimate rules silently skip). +function resetEngineState() { + clearRules(); + updateDisabledRules(null); + updateForceOverrides({ force_enable: [], force_disable: [] }); + setDisabledRuleDetails([]); +} + +beforeEach(resetEngineState); +afterEach(resetEngineState); + +describe('engine: force overrides', () => { + const diag = { check: 'UnknownFilter', message: "Unknown filter 'x'" }; + const facts = {}; + + test('force_disable beats normal enabled state', () => { + registerRule(makeRule('UnknownFilter.generic', 'UnknownFilter', 'hi')); + updateForceOverrides({ force_disable: ['UnknownFilter.generic'] }); + expect(runRules(diag, facts)).toBeNull(); + }); + + test('force_enable beats _disabledRules', () => { + registerRule(makeRule('UnknownFilter.generic', 'UnknownFilter', 'hi')); + updateDisabledRules(['UnknownFilter.generic']); + expect(runRules(diag, facts)).toBeNull(); // baseline: disabled + updateForceOverrides({ force_enable: ['UnknownFilter.generic'] }); + const r = runRules(diag, facts); + expect(r?.rule_id).toBe('UnknownFilter.generic'); + }); + + test('force_disable takes precedence over force_enable', () => { + registerRule(makeRule('UnknownFilter.generic', 'UnknownFilter', 'hi')); + updateForceOverrides({ + force_enable: ['UnknownFilter.generic'], + force_disable: ['UnknownFilter.generic'], + }); + expect(runRules(diag, facts)).toBeNull(); + }); + + test('getDisabledRuleDetails flags force_enabled rules', () => { + updateDisabledRules(['A.x']); + setDisabledRuleDetails([{ rule_id: 'A.x', effectiveness: 0.1, emitted: 10 }]); + updateForceOverrides({ force_enable: ['A.x'] }); + const details = getDisabledRuleDetails(); + expect(details).toHaveLength(1); + expect(details[0].force_enabled).toBe(true); + expect(details[0].effectiveness).toBe(0.1); + }); + + test('clearing overrides restores baseline behavior', () => { + registerRule(makeRule('UnknownFilter.generic', 'UnknownFilter', 'hi')); + updateForceOverrides({ force_disable: ['UnknownFilter.generic'] }); + expect(runRules(diag, facts)).toBeNull(); + updateForceOverrides({ force_enable: [], force_disable: [] }); + expect(runRules(diag, facts)?.rule_id).toBe('UnknownFilter.generic'); + }); +}); + +describe('engine: check-name force-disable', () => { + test('isCheckForceDisabled true only when name is in the set', () => { + updateForceOverrides({ force_disable: ['pos-supervisor:HtmlInPage'] }); + expect(isCheckForceDisabled('pos-supervisor:HtmlInPage')).toBe(true); + expect(isCheckForceDisabled('UnknownFilter')).toBe(false); + expect(isCheckForceDisabled(null)).toBe(false); + expect(isCheckForceDisabled(undefined)).toBe(false); + }); + + test('rule_ids and check names share the same force-disable set', () => { + updateForceOverrides({ force_disable: ['UnknownFilter.generic', 'pos-supervisor:HtmlInPage'] }); + expect(isCheckForceDisabled('UnknownFilter.generic')).toBe(true); + expect(isCheckForceDisabled('pos-supervisor:HtmlInPage')).toBe(true); + }); +}); diff --git a/tests/unit/rule-overrides.test.js b/tests/unit/rule-overrides.test.js new file mode 100644 index 0000000..3bad968 --- /dev/null +++ b/tests/unit/rule-overrides.test.js @@ -0,0 +1,79 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { mkdirSync, rmSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { + loadOverrides, saveOverrides, + addForceEnable, addForceDisable, removeOverride, + overrideSets, +} from '../../src/core/rule-overrides.js'; + +let projectDir; + +beforeEach(() => { + projectDir = join(tmpdir(), `pos-overrides-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}`); + mkdirSync(projectDir, { recursive: true }); +}); + +afterEach(() => { + try { rmSync(projectDir, { recursive: true, force: true }); } catch {} +}); + +describe('rule-overrides: read/write', () => { + test('loadOverrides on missing file returns empty state', () => { + const s = loadOverrides(projectDir); + expect(s.force_enable).toEqual({}); + expect(s.force_disable).toEqual({}); + }); + + test('addForceEnable persists with ts + reason', () => { + const s = addForceEnable(projectDir, 'Foo.bar', 'testing'); + expect(s.force_enable['Foo.bar'].reason).toBe('testing'); + expect(typeof s.force_enable['Foo.bar'].ts).toBe('string'); + + // Round-trip: re-read from disk. + const loaded = loadOverrides(projectDir); + expect(loaded.force_enable['Foo.bar'].reason).toBe('testing'); + }); + + test('force_enable and force_disable are mutually exclusive', () => { + addForceDisable(projectDir, 'Foo.bar', 'kill'); + const s = addForceEnable(projectDir, 'Foo.bar', 'unkill'); + expect(s.force_enable['Foo.bar']).toBeDefined(); + expect(s.force_disable['Foo.bar']).toBeUndefined(); + }); + + test('removeOverride clears both kinds', () => { + addForceEnable(projectDir, 'A.x'); + addForceDisable(projectDir, 'B.y'); + const s = removeOverride(projectDir, 'A.x'); + expect(s.force_enable['A.x']).toBeUndefined(); + expect(s.force_disable['B.y']).toBeDefined(); + }); + + test('malformed JSON file → empty state (never throws)', () => { + const path = join(projectDir, '.pos-supervisor', 'rule-overrides.json'); + mkdirSync(join(projectDir, '.pos-supervisor'), { recursive: true }); + writeFileSync(path, '{not json'); + let logged = null; + const s = loadOverrides(projectDir, { log: (m) => { logged = m; } }); + expect(s.force_enable).toEqual({}); + expect(s.force_disable).toEqual({}); + expect(logged).toContain('failed to parse'); + }); + + test('force_enable or force_disable not object → empty state', () => { + const path = join(projectDir, '.pos-supervisor', 'rule-overrides.json'); + mkdirSync(join(projectDir, '.pos-supervisor'), { recursive: true }); + writeFileSync(path, JSON.stringify({ version: 1, force_enable: 'lol', force_disable: {} })); + const s = loadOverrides(projectDir); + expect(s.force_enable).toEqual({}); + }); + + test('overrideSets converts object maps to Sets', () => { + const state = { force_enable: { 'A.x': {} }, force_disable: { 'B.y': {} } }; + const { force_enable, force_disable } = overrideSets(state); + expect(force_enable.has('A.x')).toBe(true); + expect(force_disable.has('B.y')).toBe(true); + }); +}); diff --git a/tests/unit/rules/DeprecatedTag.test.js b/tests/unit/rules/DeprecatedTag.test.js new file mode 100644 index 0000000..31dcab5 --- /dev/null +++ b/tests/unit/rules/DeprecatedTag.test.js @@ -0,0 +1,122 @@ +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/DeprecatedTag.js'; + +beforeEach(() => { clearRules(); registerRules(rules); }); + +describe('DeprecatedTag rule (upstream LSP)', () => { + test('include subrule fires on params.tag', () => { + const result = runRules({ + check: 'DeprecatedTag', + params: { tag: 'include', replacement: 'render' }, + message: "Deprecated tag 'include': replaced by render", + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.include'); + expect(result.hint_md).toContain('isolated scope'); + expect(result.fixes[0].description).toContain('render'); + }); + + test('hash_assign subrule fires on params.tag', () => { + const result = runRules({ + check: 'DeprecatedTag', + params: { tag: 'hash_assign' }, + message: "Deprecated tag 'hash_assign'", + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.hash_assign'); + expect(result.hint_md).toContain('assign x["key"]'); + }); + + test('parse_json subrule fires on params.tag', () => { + const result = runRules({ + check: 'DeprecatedTag', + params: { tag: 'parse_json' }, + message: "Deprecated tag 'parse_json'", + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.parse_json'); + expect(result.hint_md).toContain('| parse_json'); + expect(result.hint_md).toContain('capture'); + }); + + test('falls through to default for unknown deprecated tag', () => { + const result = runRules({ + check: 'DeprecatedTag', + params: { tag: 'foobar' }, + message: "Deprecated tag 'foobar'", + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.default'); + }); + + test('default rule reads tag/replacement from params when present', () => { + const result = runRules({ + check: 'DeprecatedTag', + params: { tag: 'foobar', replacement: 'baz' }, + message: 'irrelevant', + }, {}); + expect(result.hint_md).toContain('`{% foobar %}`'); + expect(result.hint_md).toContain('Use `{% baz %}` instead.'); + }); +}); + +describe('DeprecatedTag rule (pos-supervisor structural variant)', () => { + test('include subrule fires on raw message even without params.tag', () => { + // structural-warnings emits messages WITHOUT populating params.tag + // (no extractor for `pos-supervisor:DeprecatedTag` in diagnostic-record). + // The raw-message gate must catch it. + const result = runRules({ + check: 'pos-supervisor:DeprecatedTag', + params: {}, + message: '`{% include %}` is deprecated. Use `{% render %}` instead — render has isolated scope.', + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.include'); + }); + + test('hash_assign subrule fires on raw message', () => { + const result = runRules({ + check: 'pos-supervisor:DeprecatedTag', + params: {}, + message: '`{% hash_assign %}` is deprecated. Use `{% assign var["key"] = "value" %}`.', + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.hash_assign'); + }); + + test('parse_json subrule fires on raw message', () => { + const result = runRules({ + check: 'pos-supervisor:DeprecatedTag', + params: {}, + message: '`{% parse_json %}` is deprecated.', + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.parse_json'); + }); + + test('default fires when no known tag matches', () => { + const result = runRules({ + check: 'pos-supervisor:DeprecatedTag', + params: {}, + message: '`{% future_tag %}` is deprecated.', + }, {}); + expect(result.rule_id).toBe('DeprecatedTag.default'); + }); +}); + +describe('DeprecatedTag rule — guidance-only fix policy', () => { + test('every subrule emits a single guidance fix (heuristic owns text_edit)', () => { + for (const tag of ['include', 'hash_assign', 'parse_json']) { + const result = runRules({ + check: 'DeprecatedTag', + params: { tag }, + message: `Deprecated tag '${tag}'`, + }, {}); + expect(result.fixes).toHaveLength(1); + expect(result.fixes[0].type).toBe('guidance'); + } + }); + + test('default fallback emits no fix — no actionable next step without a known tag', () => { + const result = runRules({ + check: 'DeprecatedTag', + params: { tag: 'foobar' }, + message: 'irrelevant', + }, {}); + expect(result.fixes).toEqual([]); + }); +}); diff --git a/tests/unit/rules/DuplicateFunctionArguments.test.js b/tests/unit/rules/DuplicateFunctionArguments.test.js new file mode 100644 index 0000000..2557d0d --- /dev/null +++ b/tests/unit/rules/DuplicateFunctionArguments.test.js @@ -0,0 +1,38 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/DuplicateFunctionArguments.js'; + +beforeEach(() => { clearRules(); registerRules(rules); }); +afterEach(() => { clearRules(); }); + +function diag(extra = {}) { + return { + check: 'DuplicateFunctionArguments', + params: { argument: 'foo', tag_kind: 'function', partial: 'helpers/can_do', ...extra }, + message: '', + }; +} + +describe('DuplicateFunctionArguments.default', () => { + test('attribution + hint name the argument and partial for function tag', () => { + const r = runRules(diag(), {}); + expect(r.rule_id).toBe('DuplicateFunctionArguments.default'); + expect(r.hint_md).toMatch(/`foo`/); + expect(r.hint_md).toMatch(/helpers\/can_do/); + expect(r.hint_md).toMatch(/\{% function/); + expect(r.confidence).toBe(0.9); + }); + + test('render variant surfaces the right tag in the hint', () => { + const r = runRules(diag({ tag_kind: 'render', partial: 'forms/login', argument: 'email' }), {}); + expect(r.hint_md).toMatch(/`email`/); + expect(r.hint_md).toMatch(/\{% render/); + expect(r.hint_md).toMatch(/forms\/login/); + }); + + test('falls back to safe wording when params are missing', () => { + const r = runRules({ check: 'DuplicateFunctionArguments', message: '' }, {}); + expect(r.rule_id).toBe('DuplicateFunctionArguments.default'); + expect(r.hint_md).toMatch(/duplicate argument/i); + }); +}); diff --git a/tests/unit/rules/InvalidLayout.test.js b/tests/unit/rules/InvalidLayout.test.js new file mode 100644 index 0000000..884329d --- /dev/null +++ b/tests/unit/rules/InvalidLayout.test.js @@ -0,0 +1,183 @@ +// InvalidLayout end-to-end coverage: +// 1. structural emitter detects the project's layout-extension convention +// and bakes the right `Expected file:` path into the message. +// 2. fix-generator's `extractLayoutPath` lifts that path verbatim. +// 3. The new `InvalidLayout.default` rule attaches a stable rule_id + +// create_file fix that lands at the correct path. +// 4. `suppressUpstreamFrontmatterDup` drops the upstream +// `ValidFrontmatter.layout_missing` even when its line diverges from +// our structural emission — matched by layout NAME, not just line. + +import { describe, test, expect, beforeAll, beforeEach, afterAll } from 'bun:test'; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules as InvalidLayoutRules } from '../../../src/core/rules/InvalidLayout.js'; +import { generateStructuralWarnings } from '../../../src/core/structural-warnings.js'; +import { suppressUpstreamFrontmatterDup } from '../../../src/core/diagnostic-pipeline.js'; +import { parseLiquidFile, extractAllFromAST } from '../../../src/core/liquid-parser.js'; + +beforeEach(() => { clearRules(); registerRules(InvalidLayoutRules); }); + +function emit(projectDir, content, file = 'app/views/pages/x.liquid') { + const ast = parseLiquidFile(content); + const structural = extractAllFromAST(ast); + return generateStructuralWarnings(ast, content, file, structural, new Set(), { projectDir }); +} + +describe('InvalidLayout — structural emitter detects layout extension', () => { + let dir; + beforeAll(() => { + dir = mkdtempSync(join(tmpdir(), 'invalid-layout-')); + mkdirSync(join(dir, 'app/views/layouts'), { recursive: true }); + // Project uses BARE .liquid (DEMO convention). + writeFileSync(join(dir, 'app/views/layouts/application.liquid'), '{{ content_for_layout }}'); + }); + afterAll(() => { rmSync(dir, { recursive: true, force: true }); }); + + test('emitter picks `.liquid` extension when project uses bare suffix', () => { + const ws = emit(dir, '---\nlayout: nonexistent\n---\n

x

'); + const inv = ws.find(w => w.check === 'pos-supervisor:InvalidLayout'); + expect(inv).toBeDefined(); + expect(inv.message).toContain('app/views/layouts/nonexistent.liquid'); + expect(inv.message).not.toContain('nonexistent.html.liquid'); + }); + + test('rule lifts the corrected path into the create_file fix', () => { + const ws = emit(dir, '---\nlayout: nonexistent\n---\n

x

'); + const inv = ws.find(w => w.check === 'pos-supervisor:InvalidLayout'); + const r = runRules({ check: inv.check, message: inv.message }, {}); + expect(r.rule_id).toBe('InvalidLayout.default'); + expect(r.fixes[0].type).toBe('create_file'); + expect(r.fixes[0].path).toBe('app/views/layouts/nonexistent.liquid'); + expect(r.fixes[0].path).not.toContain('html.liquid'); + }); +}); + +describe('InvalidLayout — emitter picks `.html.liquid` when project uses it', () => { + let dir; + beforeAll(() => { + dir = mkdtempSync(join(tmpdir(), 'invalid-layout-html-')); + mkdirSync(join(dir, 'app/views/layouts'), { recursive: true }); + writeFileSync(join(dir, 'app/views/layouts/application.html.liquid'), '{{ content_for_layout }}'); + }); + afterAll(() => { rmSync(dir, { recursive: true, force: true }); }); + + test('extension matches the existing convention', () => { + const ws = emit(dir, '---\nlayout: nonexistent\n---\n

x

'); + const inv = ws.find(w => w.check === 'pos-supervisor:InvalidLayout'); + expect(inv.message).toContain('nonexistent.html.liquid'); + }); +}); + +describe('InvalidLayout — defaults to `.liquid` when layouts dir is empty', () => { + let dir; + beforeAll(() => { + dir = mkdtempSync(join(tmpdir(), 'invalid-layout-empty-')); + mkdirSync(join(dir, 'app/views/layouts'), { recursive: true }); + }); + afterAll(() => { rmSync(dir, { recursive: true, force: true }); }); + + test('no existing layouts → bare suffix (modern convention)', () => { + const ws = emit(dir, '---\nlayout: app\n---\n

x

'); + const inv = ws.find(w => w.check === 'pos-supervisor:InvalidLayout'); + expect(inv.message).toContain('app/views/layouts/app.liquid'); + }); +}); + +describe('suppressUpstreamFrontmatterDup — by layout name, line-independent', () => { + test('drops ValidFrontmatter `layout_missing` when InvalidLayout names same layout, even on different lines', () => { + const result = { + errors: [], + warnings: [ + { + check: 'pos-supervisor:InvalidLayout', + severity: 'warning', + message: 'Layout `application` not found. Expected file: `app/views/layouts/application.liquid`.', + line: 2, + column: 0, + }, + { + check: 'ValidFrontmatter', + severity: 'warning', + message: "Layout 'application' does not exist", + line: 99, // intentionally NOT 2 — would survive line-only dedup + column: 0, + }, + ], + infos: [], + }; + const removed = suppressUpstreamFrontmatterDup(result); + expect(removed).toBe(1); + expect(result.warnings.map(w => w.check)).toEqual(['pos-supervisor:InvalidLayout']); + expect(result.infos[0].message).toContain('Suppressed 1 ValidFrontmatter'); + }); + + test('keeps unrelated ValidFrontmatter (different layout name)', () => { + const result = { + errors: [], + warnings: [ + { check: 'pos-supervisor:InvalidLayout', message: 'Layout `application` not found. Expected file: `app/views/layouts/application.liquid`.', line: 2 }, + { check: 'ValidFrontmatter', message: "Layout 'other_layout' does not exist", line: 5 }, + ], + infos: [], + }; + suppressUpstreamFrontmatterDup(result); + expect(result.warnings.map(w => w.check)).toEqual(['pos-supervisor:InvalidLayout', 'ValidFrontmatter']); + }); + + test('keeps non-layout ValidFrontmatter categories (deprecated_field, missing_required, etc.)', () => { + const result = { + errors: [], + warnings: [ + { check: 'pos-supervisor:InvalidLayout', message: 'Layout `application` not found. Expected file: `app/views/layouts/application.liquid`.', line: 2 }, + { check: 'ValidFrontmatter', message: 'Missing required frontmatter field `slug` in Page file', line: 1 }, + { check: 'ValidFrontmatter', message: '`layout_name` is deprecated. Use `layout` instead.', line: 3 }, + ], + infos: [], + }; + suppressUpstreamFrontmatterDup(result); + expect(result.warnings.map(w => w.message.slice(0, 30))).toEqual([ + 'Layout `application` not found', + 'Missing required frontmatter f', + '`layout_name` is deprecated. U', + ]); + }); + + test('still dedups on line match when layout-name match is unavailable', () => { + // pos-supervisor:InvalidFrontMatter (NOT InvalidLayout) — line is the + // only signal the dedup has for this pair. + const result = { + errors: [], + warnings: [ + { check: 'pos-supervisor:InvalidFrontMatter', message: 'Unknown front-matter key', line: 4 }, + { check: 'ValidFrontmatter', message: 'Unknown frontmatter field `weird` in Page file', line: 4 }, + ], + infos: [], + }; + const removed = suppressUpstreamFrontmatterDup(result); + expect(removed).toBe(1); + expect(result.warnings.map(w => w.check)).toEqual(['pos-supervisor:InvalidFrontMatter']); + }); +}); + +describe('InvalidLayout rule — defensive paths', () => { + test('falls back to guidance when message lacks the Expected-file clause', () => { + const r = runRules({ + check: 'pos-supervisor:InvalidLayout', + message: 'Layout `app` was not found.', // no `Expected file:` clause + }, {}); + expect(r.rule_id).toBe('InvalidLayout.default'); + expect(r.fixes[0].type).toBe('guidance'); + }); + + test('see_also points at layouts domain guide', () => { + const r = runRules({ + check: 'pos-supervisor:InvalidLayout', + message: 'Layout `app` not found. Expected file: `app/views/layouts/app.liquid`.', + }, {}); + expect(r.see_also.tool).toBe('domain_guide'); + expect(r.see_also.args.domain).toBe('layouts'); + }); +}); diff --git a/tests/unit/rules/JsonLiteralQuoteStyle.test.js b/tests/unit/rules/JsonLiteralQuoteStyle.test.js new file mode 100644 index 0000000..4bab886 --- /dev/null +++ b/tests/unit/rules/JsonLiteralQuoteStyle.test.js @@ -0,0 +1,16 @@ +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/JsonLiteralQuoteStyle.js'; + +beforeEach(() => { clearRules(); registerRules(rules); }); +afterEach(() => { clearRules(); }); + +describe('JsonLiteralQuoteStyle.default', () => { + test('attributes every emit (single-shot rule)', () => { + const r = runRules({ check: 'JsonLiteralQuoteStyle', params: {}, message: '' }, {}); + expect(r.rule_id).toBe('JsonLiteralQuoteStyle.default'); + expect(r.hint_md).toMatch(/double-quoted/); + expect(r.hint_md).toMatch(/JSON literal/); + expect(r.confidence).toBe(0.95); + }); +}); diff --git a/tests/unit/rules/MissingPartial.test.js b/tests/unit/rules/MissingPartial.test.js new file mode 100644 index 0000000..2fef604 --- /dev/null +++ b/tests/unit/rules/MissingPartial.test.js @@ -0,0 +1,400 @@ +import { describe, test, expect, beforeEach, beforeAll, afterAll } from 'bun:test'; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules, parseModulePath } from '../../../src/core/rules/MissingPartial.js'; +import { buildFactGraph } from '../../../src/core/project-fact-graph.js'; + +const FIXTURE_MAP = { + pages: { + 'blog_posts:get': { path: 'app/views/pages/blog_posts/index.html.liquid', slug: 'blog_posts', method: 'get', renders: ['blog_posts/list'], function_calls: [] }, + }, + partials: { + 'blog_posts/list': { path: 'app/views/partials/blog_posts/list.liquid', params: [], renders: ['blog_posts/card'], function_calls: [], rendered_by: [] }, + 'blog_posts/card': { path: 'app/views/partials/blog_posts/card.liquid', params: [], renders: [], function_calls: [], rendered_by: [] }, + 'blog_posts/form': { path: 'app/views/partials/blog_posts/form.liquid', params: [], renders: [], function_calls: [], rendered_by: [] }, + }, + commands: {}, + queries: {}, + graphql: {}, + schema: {}, + layouts: {}, + translations: {}, + assets: [], +}; + +const graph = buildFactGraph(FIXTURE_MAP); +const facts = { graph }; + +beforeEach(() => { + clearRules(); + registerRules(rules); +}); + +describe('MissingPartial.module_path', () => { + test('fires for module partial paths', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/user/helpers/auth' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.module_path'); + expect(result.see_also.tool).toBe('module_info'); + expect(result.see_also.args.name).toBe('user'); + expect(result.confidence).toBeGreaterThanOrEqual(0.7); + }); + + test('does not fire for project partials', () => { + const diag = { check: 'MissingPartial', params: { partial: 'blog_posts/missing' } }; + const result = runRules(diag, facts); + expect(result.rule_id).not.toBe('MissingPartial.module_path'); + }); +}); + +describe('MissingPartial.module_path — projectDir-aware behavior', () => { + let projectDir; + + beforeAll(() => { + projectDir = mkdtempSync(join(tmpdir(), 'mp-modpath-')); + + const writeFile = (rel) => { + const abs = join(projectDir, rel); + mkdirSync(join(abs, '..'), { recursive: true }); + writeFileSync(abs, ''); + }; + + // core: only `execute` is exported as a command, plus a deeper helper tree + writeFile('modules/core/public/lib/commands/execute.liquid'); + writeFile('modules/core/public/lib/commands/email/send/build.liquid'); + writeFile('modules/core/public/lib/commands/email/send/check.liquid'); + writeFile('modules/core/public/lib/queries/users/find.liquid'); + writeFile('modules/core/public/lib/helpers/auth_token.liquid'); + + // user: only helpers + writeFile('modules/user/public/lib/helpers/current.liquid'); + }); + + afterAll(() => { + rmSync(projectDir, { recursive: true, force: true }); + }); + + test('build/check special case: explains they are inline phases of caller command', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/core/commands/build' } }; + const result = runRules(diag, { ...facts, projectDir }); + expect(result.rule_id).toBe('MissingPartial.module_path'); + expect(result.hint_md).toContain('inline phases of your own command'); + expect(result.hint_md).toContain('modules/core/commands/execute'); + // closest matches block must enumerate live exports + expect(result.hint_md).toContain('modules/core/commands/execute'); + // exported categories summary + expect(result.hint_md).toContain('Exported categories:'); + expect(result.hint_md).toMatch(/commands \(\d+\)/); + }); + + test('build/check special case fires for `check` symmetrically', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/core/commands/check' } }; + const result = runRules(diag, { ...facts, projectDir }); + expect(result.hint_md).toContain('inline phases of your own command'); + expect(result.fixes[0].description).toContain('inline the build/check logic'); + }); + + test('non-existing path inside an installed module: lists Levenshtein candidates', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/core/queries/users/fnd' } }; + const result = runRules(diag, { ...facts, projectDir }); + expect(result.rule_id).toBe('MissingPartial.module_path'); + expect(result.hint_md).toContain('not exported by module `core`'); + expect(result.hint_md).toContain('modules/core/queries/users/find'); + expect(result.fixes[0].description).toContain('modules/core/queries/users/find'); + expect(result.confidence).toBe(0.9); + }); + + test('module not installed: suggests the closest installed module', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/cre/commands/execute' } }; + const result = runRules(diag, { ...facts, projectDir }); + expect(result.rule_id).toBe('MissingPartial.module_path'); + expect(result.hint_md).toContain('Module `cre` is not installed'); + expect(result.hint_md).toContain('Installed modules:'); + expect(result.hint_md).toContain('Did you mean `core`'); + expect(result.see_also.tool).toBe('project_map'); + }); + + test('module not installed and no modules dir: still produces a hint', () => { + const isolatedDir = mkdtempSync(join(tmpdir(), 'mp-empty-')); + try { + const diag = { check: 'MissingPartial', params: { partial: 'modules/anything/commands/execute' } }; + const result = runRules(diag, { ...facts, projectDir: isolatedDir }); + expect(result.rule_id).toBe('MissingPartial.module_path'); + expect(result.hint_md).toContain('Module `anything` is not installed'); + expect(result.hint_md).toContain('No modules are installed'); + } finally { + rmSync(isolatedDir, { recursive: true, force: true }); + } + }); + + test('no projectDir in facts: rule still fires with degraded hint (no exports)', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/core/commands/build' } }; + const result = runRules(diag, facts); // no projectDir + expect(result.rule_id).toBe('MissingPartial.module_path'); + expect(result.hint_md).toContain('inline phases of your own command'); + // no live exports → no closest matches + expect(result.hint_md).toContain('(no close matches in this module)'); + }); + + test('parseModulePath: splits into moduleName / category / rest', () => { + expect(parseModulePath('modules/core/commands/email/send/build')) + .toEqual({ moduleName: 'core', category: 'commands', rest: 'email/send/build' }); + expect(parseModulePath('modules/core/commands/build')) + .toEqual({ moduleName: 'core', category: 'commands', rest: 'build' }); + expect(parseModulePath('modules/core/commands')) + .toEqual({ moduleName: 'core', category: 'commands', rest: null }); + expect(parseModulePath('modules/core')) + .toEqual({ moduleName: 'core', category: null, rest: null }); + expect(parseModulePath('')) + .toEqual({ moduleName: null, category: null, rest: null }); + expect(parseModulePath('app/lib/commands/foo')) + .toEqual({ moduleName: null, category: null, rest: null }); + }); +}); + +describe('MissingPartial.file_exists', () => { + test('fires when target file exists in graph', () => { + const diag = { check: 'MissingPartial', params: { partial: 'blog_posts/card' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.file_exists'); + expect(result.hint_md).toContain('exists'); + }); +}); + +describe('MissingPartial.suggest_nearest', () => { + test('suggests similar partials for near-miss names', () => { + const diag = { check: 'MissingPartial', params: { partial: 'blog_posts/lst' }, file: 'app/views/pages/blog_posts/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.suggest_nearest'); + expect(result.hint_md).toContain('blog_posts/list'); + }); + + test('skips for module paths', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/user/lst' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.module_path'); + }); +}); + +describe('MissingPartial.create_file', () => { + test('suggests creating a new partial', () => { + const diag = { check: 'MissingPartial', params: { partial: 'invoices/summary' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.create_file'); + expect(result.fixes).toHaveLength(1); + expect(result.fixes[0].type).toBe('create_file'); + expect(result.fixes[0].path).toBe('app/views/partials/invoices/summary.liquid'); + }); + + test('suggests creating a command', () => { + const diag = { check: 'MissingPartial', params: { partial: 'commands/products/create' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.create_file'); + expect(result.fixes[0].path).toBe('app/lib/commands/products/create.liquid'); + }); + + test('suggests creating a query', () => { + const diag = { check: 'MissingPartial', params: { partial: 'queries/products/find' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.create_file'); + expect(result.fixes[0].path).toBe('app/lib/queries/products/find.liquid'); + }); + + test('does not suggest create for existing files', () => { + const diag = { check: 'MissingPartial', params: { partial: 'blog_posts/card' } }; + const result = runRules(diag, facts); + expect(result.rule_id).not.toBe('MissingPartial.create_file'); + }); + + test('does not suggest create for module paths', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/core/commands/execute' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.module_path'); + }); +}); + +describe('MissingPartial.invalid_lib_prefix', () => { + // Regression: prior code stripped a leading `lib/` everywhere it saw one, + // collapsing `lib/commands/X` and `commands/X` into the same bucket. That + // hid the bug from agents (and from us) — `lib/commands/X` is *not* a + // valid platformOS function-tag path, since paths resolve under + // `app/views/partials/` and `app/lib/`. The literal prefix expands to + // `app/lib/lib/...` and never resolves. + + test('fires for `lib/commands/X` with the corrected name in the hint', () => { + const diag = { + check: 'MissingPartial', + params: { partial: 'lib/commands/contact_submissions/create' }, + line: 6, + column: 20, + endLine: 6, + endColumn: 61, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.invalid_lib_prefix'); + expect(result.hint_md).toContain('lib/commands/contact_submissions/create'); + expect(result.hint_md).toContain('commands/contact_submissions/create'); + expect(result.confidence).toBeGreaterThanOrEqual(0.9); + }); + + test('emits a text_edit fix that replaces the quoted reference with the corrected form', () => { + const diag = { + check: 'MissingPartial', + params: { partial: 'lib/commands/contact_submissions/create' }, + line: 6, + column: 20, + endLine: 6, + endColumn: 61, + }; + const result = runRules(diag, facts); + expect(result.fixes).toHaveLength(1); + const fix = result.fixes[0]; + expect(fix.type).toBe('text_edit'); + expect(fix.new_text).toBe(`'commands/contact_submissions/create'`); + expect(fix.range).toEqual({ + start: { line: 6, character: 20 }, + end: { line: 6, character: 61 }, + }); + }); + + test('falls back to a guidance fix when the diagnostic lacks position fields', () => { + const diag = { + check: 'MissingPartial', + params: { partial: 'lib/queries/products/search' }, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.invalid_lib_prefix'); + expect(result.fixes).toHaveLength(1); + expect(result.fixes[0].type).toBe('guidance'); + expect(result.fixes[0].description).toContain('lib/queries/products/search'); + expect(result.fixes[0].description).toContain('queries/products/search'); + }); + + test('handles `lib/queries/X` symmetrically with `lib/commands/X`', () => { + const diag = { + check: 'MissingPartial', + params: { partial: 'lib/queries/products/search' }, + line: 4, + column: 16, + endLine: 4, + endColumn: 47, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.invalid_lib_prefix'); + expect(result.fixes[0].type).toBe('text_edit'); + expect(result.fixes[0].new_text).toBe(`'queries/products/search'`); + }); + + test('does NOT fire for the bare `commands/X` form (the canonical syntax)', () => { + const diag = { + check: 'MissingPartial', + params: { partial: 'commands/contact_submissions/create' }, + line: 1, column: 0, endLine: 1, endColumn: 35, + }; + const result = runRules(diag, facts); + expect(result?.rule_id).not.toBe('MissingPartial.invalid_lib_prefix'); + }); + + test('does NOT fire for module paths that happen to contain `lib/`', () => { + // Module paths look like `modules/core/lib/commands/...` in some tree + // layouts on disk, but the *call* path is `modules//...` — never + // begins with `lib/`. Guard against false positives. + const diag = { + check: 'MissingPartial', + params: { partial: 'modules/core/commands/execute' }, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.module_path'); + }); + + test('beats lower-priority rules: invalid_lib_prefix wins over create_file even when the corrected file would not exist', () => { + // The `lib/`-stripped path `commands/never/written` resolves to + // `app/lib/commands/never/written.liquid` — absent from the fact graph. + // create_file would happily propose creating it; the prefix rule must + // fire first so the agent is told to fix the path, not create a phantom. + const diag = { + check: 'MissingPartial', + params: { partial: 'lib/commands/never/written' }, + line: 2, column: 10, endLine: 2, endColumn: 39, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.invalid_lib_prefix'); + }); +}); + +describe('MissingPartial — edge cases', () => { + test('falls through to .default when partial param is missing (was .unmatched before catch-all)', () => { + const diag = { check: 'MissingPartial', params: {} }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MissingPartial.default'); + // No symbol name in the hint when extraction failed. + expect(result.hint_md).toContain('this reference'); + expect(result.hint_md).not.toContain('``'); // no empty backticked symbol + }); + + test('falls through to .default when params is undefined', () => { + const diag = { check: 'MissingPartial' }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('MissingPartial.default'); + }); + + test('rule_id is always set in result', () => { + const diag = { check: 'MissingPartial', params: { partial: 'invoices/receipt' } }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBeDefined(); + expect(result.rule_id.startsWith('MissingPartial.')).toBe(true); + }); +}); + +describe('MissingPartial.default catch-all', () => { + test('does NOT preempt a more specific rule (priority order intact)', () => { + const diag = { + check: 'MissingPartial', + params: { partial: 'lib/commands/orders/create' }, + line: 3, column: 12, endLine: 3, endColumn: 41, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.invalid_lib_prefix'); + }); + + test('does NOT preempt MissingPartial.module_path', () => { + const diag = { check: 'MissingPartial', params: { partial: 'modules/user/helpers/auth' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.module_path'); + }); + + test('emits the partial name in the hint when extractParams found one', () => { + const diag = { + check: 'MissingPartial', + // Path that doesn't match any specialised guard: not modules/, not lib/, + // not file_exists, no levenshtein neighbours, classifyPath says nothing + // useful → .default catches it. + params: { partial: 'completely/unrecognisable/shape/that/no/specialised/rule/wants' }, + }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + // Either a more specific rule fires (acceptable) or the default carries + // the symbol name. The contract is "every emit gets a typed rule_id". + expect(result.rule_id.startsWith('MissingPartial.')).toBe(true); + expect(result.rule_id).not.toBe('MissingPartial.unmatched'); + }); + + test('default rule confidence is intentionally lower than the named guards', () => { + const diag = { check: 'MissingPartial' }; // no params at all + const result = runRules(diag, facts); + expect(result.rule_id).toBe('MissingPartial.default'); + expect(result.confidence).toBeLessThanOrEqual(0.6); + }); + + test('default emits a project_map see_also so the agent has a recovery path', () => { + const diag = { check: 'MissingPartial', params: {} }; + const result = runRules(diag, facts); + expect(result.see_also).toBeDefined(); + expect(result.see_also.tool).toBe('project_map'); + }); +}); diff --git a/tests/unit/rules/NonGetRenderingPage.test.js b/tests/unit/rules/NonGetRenderingPage.test.js new file mode 100644 index 0000000..c89f0a0 --- /dev/null +++ b/tests/unit/rules/NonGetRenderingPage.test.js @@ -0,0 +1,187 @@ +// NonGetRenderingPage three-subrule routing — covers each path through +// `validatePageMethodAndForms` (structural-warnings.js) end-to-end into the +// rule engine. Test cases mirror the gist analysis at +// docs/rule-performance-plan.md / NonGetRenderingPageRule.md. + +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/NonGetRenderingPage.js'; +import { generateStructuralWarnings } from '../../../src/core/structural-warnings.js'; +import { parseLiquidFile, extractAllFromAST } from '../../../src/core/liquid-parser.js'; + +beforeEach(() => { clearRules(); registerRules(rules); }); + +function emit(file, content) { + const ast = parseLiquidFile(content); + const structural = extractAllFromAST(ast); + const ws = generateStructuralWarnings(ast, content, file, structural, new Set(), {}); + return ws.filter(w => w.check === 'pos-supervisor:NonGetRenderingPage'); +} + +function route(diag) { + return runRules({ ...diag }, {}); +} + +describe('NonGetRenderingPage.html_on_post', () => { + test('non-API POST page with layout fires html_on_post', () => { + const ws = emit('app/views/pages/contact.liquid', + '---\nslug: contact\nmethod: post\nlayout: application\n---\n

Contact

'); + expect(ws).toHaveLength(1); + const r = route(ws[0]); + expect(r.rule_id).toBe('NonGetRenderingPage.html_on_post'); + expect(r.fixes[0].description).toContain('landing page'); + expect(r.fixes[0].description).toContain('API handler'); + }); + + test('hint references both UI-page and API-handler shapes', () => { + const ws = emit('app/views/pages/contact.liquid', + '---\nslug: contact\nmethod: post\nlayout: application\n---\n

x

'); + const r = route(ws[0]); + expect(r.hint_md).toContain('Landing / display page'); + expect(r.hint_md).toContain('Form-handling endpoint'); + expect(r.hint_md).toContain("action=\"/api/contacts/create\""); + }); + + test('non-API PUT/DELETE/PATCH pages with HTML also fire', () => { + for (const method of ['put', 'delete', 'patch']) { + const ws = emit('app/views/pages/x.liquid', + `---\nslug: x\nmethod: ${method}\nlayout: application\n---\n

x

`); + expect(ws).toHaveLength(1); + const r = route(ws[0]); + expect(r.rule_id).toBe('NonGetRenderingPage.html_on_post'); + expect(r.hint_md).toContain(`method: ${method}`); + } + }); + + test('POST page with no HTML (redirect-only handler) does NOT fire', () => { + const ws = emit('app/views/pages/contacts.liquid', + '---\nslug: contacts\nmethod: post\n---\n{% graphql r = "contacts/create" %}'); + expect(ws).toEqual([]); + }); +}); + +describe('NonGetRenderingPage.api_renders_html', () => { + test('API page with layout fires api_renders_html', () => { + const ws = emit('app/views/pages/api/contacts/create.liquid', + '---\nslug: api/contacts/create\nmethod: post\nlayout: application\n---\n

Creating

'); + expect(ws).toHaveLength(1); + const r = route(ws[0]); + expect(r.rule_id).toBe('NonGetRenderingPage.api_renders_html'); + expect(r.hint_md).toContain('format: json'); + expect(r.hint_md).toContain('result | json'); + }); + + test('API page missing format:json fires even without HTML body', () => { + const ws = emit('app/views/pages/api/contacts/create.liquid', + '---\nslug: api/contacts/create\nmethod: post\n---\n{% graphql r = "contacts/create" %}\n{{ r | json }}'); + expect(ws).toHaveLength(1); + const r = route(ws[0]); + expect(r.rule_id).toBe('NonGetRenderingPage.api_renders_html'); + expect(r.hint_md).toContain('format: json'); + }); + + test('valid API endpoint with format:json + json body emits NOTHING', () => { + const ws = emit('app/views/pages/api/contacts/create.liquid', + '---\nslug: api/contacts/create\nmethod: post\nformat: json\n---\n{% graphql r = "contacts/create" %}\n{{ r | json }}'); + expect(ws).toEqual([]); + }); + + test('/_/ and /internal/ prefixes are also API paths', () => { + for (const prefix of ['_', 'internal']) { + const ws = emit(`app/views/pages/${prefix}/x.liquid`, + `---\nslug: ${prefix}/x\nmethod: post\nlayout: application\n---\n

x

`); + expect(ws).toHaveLength(1); + const r = route(ws[0]); + expect(r.rule_id).toBe('NonGetRenderingPage.api_renders_html'); + } + }); + + test('extracted slug appears in hint canonical-shape example', () => { + const ws = emit('app/views/pages/api/foo/bar.liquid', + '---\nslug: api/foo/bar\nmethod: put\nlayout: application\n---\n

x

'); + const r = route(ws[0]); + expect(r.hint_md).toContain('slug: api/foo/bar'); + expect(r.hint_md).toContain('method: put'); + }); +}); + +describe('NonGetRenderingPage.get_form_target', () => { + test('GET page with form to non-API path fires get_form_target', () => { + const ws = emit('app/views/pages/index.liquid', + '---\nslug: index\n---\n
'); + expect(ws).toHaveLength(1); + const r = route(ws[0]); + expect(r.rule_id).toBe('NonGetRenderingPage.get_form_target'); + expect(r.hint_md).toContain('/api/contacts/create'); + expect(r.hint_md).toContain('app/views/pages/api/contacts/create.liquid'); + }); + + test('form action under /api/ is sanctioned — no diagnostic', () => { + const ws = emit('app/views/pages/index.liquid', + '---\nslug: index\n---\n
'); + expect(ws).toEqual([]); + }); + + test('self-posting form (action == own slug) is sanctioned — no diagnostic', () => { + const ws = emit('app/views/pages/contacts.liquid', + '---\nslug: contacts\n---\n
'); + expect(ws).toEqual([]); + }); + + test('GET form (method="get") is not flagged — no submission risk', () => { + const ws = emit('app/views/pages/search.liquid', + '---\nslug: search\n---\n
'); + expect(ws).toEqual([]); + }); + + test('attribute order is irrelevant — action before method works', () => { + const ws = emit('app/views/pages/index.liquid', + '---\nslug: index\n---\n
'); + expect(ws).toHaveLength(1); + expect(route(ws[0]).rule_id).toBe('NonGetRenderingPage.get_form_target'); + }); + + test('multiple forms — emits one diagnostic per offending form', () => { + const ws = emit('app/views/pages/index.liquid', + '---\nslug: index\n---\n
\n
\n
'); + expect(ws).toHaveLength(2); + const actions = ws.map(w => w.message.match(/posts to `([^`]+)`/)?.[1]); + expect(actions).toEqual(['/a', '/c']); + }); + + test('form with single quotes parses correctly', () => { + const ws = emit('app/views/pages/index.liquid', + "---\nslug: index\n---\n
"); + expect(ws).toHaveLength(1); + expect(route(ws[0]).rule_id).toBe('NonGetRenderingPage.get_form_target'); + }); +}); + +describe('NonGetRenderingPage default fallback', () => { + test('unknown subtype message routes to default rule', () => { + const r = runRules({ + check: 'pos-supervisor:NonGetRenderingPage', + message: 'Some new diagnostic shape we have not seen', + }, {}); + expect(r.rule_id).toBe('NonGetRenderingPage.default'); + expect(r.confidence).toBeLessThanOrEqual(0.6); + }); +}); + +describe('NonGetRenderingPage — DEMO regression cases', () => { + test('the original DEMO failure (POST landing page) now ships actionable fix', () => { + // Pre-task-4 the rule was `NonGetRenderingPage.default` with `fixes: []` + // and 25 outcomes (5 resolved / 15 unchanged / 5 regressed). + const ws = emit('app/views/pages/contact.liquid', + '---\nslug: contact\nmethod: post\nlayout: application\n---\n

Contact

\n
...
'); + expect(ws).toHaveLength(1); + const r = route(ws[0]); + expect(r.rule_id).toBe('NonGetRenderingPage.html_on_post'); + expect(r.fixes).toHaveLength(1); + expect(r.fixes[0].type).toBe('guidance'); + // Hint disambiguates the two valid intents (landing vs API handler) so + // the agent's loop-on-unchanged behaviour stops. + expect(r.hint_md).toContain('Landing / display page'); + expect(r.hint_md).toContain('Form-handling endpoint'); + }); +}); diff --git a/tests/unit/rules/Tier1Rules.test.js b/tests/unit/rules/Tier1Rules.test.js new file mode 100644 index 0000000..2f264a6 --- /dev/null +++ b/tests/unit/rules/Tier1Rules.test.js @@ -0,0 +1,88 @@ +/** + * Tier-1 rule modules — attribution + hint only. The text_edit fix is produced + * by fix-generator.js in full mode; these rules exist so the diagnostic gets + * a stable rule_id (instead of `.unmatched`) and a useful hint in the + * agent's response. + */ + +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules as ImgLazyLoadingRules } from '../../../src/core/rules/ImgLazyLoading.js'; +import { rules as ImgWidthAndHeightRules } from '../../../src/core/rules/ImgWidthAndHeight.js'; +import { rules as ConvertIncludeToRenderRules } from '../../../src/core/rules/ConvertIncludeToRender.js'; +import { rules as NonGetRenderingPageRules } from '../../../src/core/rules/NonGetRenderingPage.js'; + +describe('ImgLazyLoading.recommended', () => { + beforeEach(() => { clearRules(); registerRules(ImgLazyLoadingRules); }); + + test('fires on every ImgLazyLoading diagnostic with the canonical rule_id', () => { + const result = runRules( + { check: 'ImgLazyLoading', message: 'img without loading', line: 3, column: 2 }, + { graph: null }, + ); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('ImgLazyLoading.recommended'); + expect(result.confidence).toBe(0.9); + expect(result.hint_md).toMatch(/loading="lazy"/); + expect(result.fixes).toEqual([]); + }); + + test('returns null for other checks', () => { + expect(runRules({ check: 'UnknownFilter' }, { graph: null })).toBeNull(); + }); +}); + +describe('ImgWidthAndHeight.recommended', () => { + beforeEach(() => { clearRules(); registerRules(ImgWidthAndHeightRules); }); + + test('fires with canonical rule_id + CLS hint', () => { + const result = runRules( + { check: 'ImgWidthAndHeight', message: 'missing width/height', line: 5 }, + { graph: null }, + ); + expect(result.rule_id).toBe('ImgWidthAndHeight.recommended'); + expect(result.confidence).toBe(0.9); + expect(result.hint_md).toMatch(/width/i); + expect(result.hint_md).toMatch(/height/i); + }); +}); + +describe('ConvertIncludeToRender.default', () => { + beforeEach(() => { clearRules(); registerRules(ConvertIncludeToRenderRules); }); + + test('fires with canonical rule_id + explains render scope', () => { + const result = runRules( + { check: 'ConvertIncludeToRender', message: 'use render instead of include', line: 10 }, + { graph: null }, + ); + expect(result.rule_id).toBe('ConvertIncludeToRender.default'); + expect(result.confidence).toBe(0.9); + expect(result.hint_md).toMatch(/render/); + expect(result.hint_md).toMatch(/isolated scope/); + }); +}); + +describe('NonGetRenderingPage.default — fallback for non-discriminated messages', () => { + beforeEach(() => { clearRules(); registerRules(NonGetRenderingPageRules); }); + + // After the task-4 split into three subrules (html_on_post / api_renders_html + // / get_form_target) the default rule only fires when none of the + // discriminator regexes match. Subrule routing is exercised in + // tests/unit/rules/NonGetRenderingPage.test.js — this case is the + // safety net for upstream message-shape drift. + test('fires with canonical rule_id + names the three valid platformOS shapes', () => { + const result = runRules( + { check: 'pos-supervisor:NonGetRenderingPage', message: 'a brand-new diagnostic shape this rule has not seen before' }, + { graph: null }, + ); + expect(result.rule_id).toBe('NonGetRenderingPage.default'); + expect(result.confidence).toBeLessThanOrEqual(0.6); // fallback confidence is intentionally lower + expect(result.hint_md).toMatch(/UI page/); + expect(result.hint_md).toMatch(/API endpoint/); + expect(result.hint_md).toMatch(/Forms on GET pages/); + // The fallback now ships a single guidance fix (the old empty-fixes + // behaviour drove the DEMO loop-on-unchanged regression). + expect(result.fixes).toHaveLength(1); + expect(result.fixes[0].type).toBe('guidance'); + }); +}); diff --git a/tests/unit/rules/Tier3Rules.test.js b/tests/unit/rules/Tier3Rules.test.js new file mode 100644 index 0000000..679733f --- /dev/null +++ b/tests/unit/rules/Tier3Rules.test.js @@ -0,0 +1,185 @@ +// Tier 3 promotion tests — every Bucket B `.unmatched` check that gained a +// rule module in task 3 phase 1. Scope: stable rule_id, structured hint, +// guidance fix that doesn't compete with the existing fix-generator +// heuristic (where one exists). Each describe block targets one check. + +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; + +import { rules as UnrecognizedRules } from '../../../src/core/rules/UnrecognizedRenderPartialArguments.js'; +import { rules as SchemaPropertyRules } from '../../../src/core/rules/SchemaProperty.js'; +import { rules as SchemaYAMLRules } from '../../../src/core/rules/SchemaYAML.js'; +import { rules as MissingSlugRules } from '../../../src/core/rules/MissingSlug.js'; +import { rules as MissingContentRules } from '../../../src/core/rules/MissingContentForLayout.js'; +import { rules as ParserBlockingRules } from '../../../src/core/rules/ParserBlockingScript.js'; +import { rules as TranslationLocaleRules } from '../../../src/core/rules/TranslationMissingLocaleKey.js'; + +describe('UnrecognizedRenderPartialArguments rule', () => { + beforeEach(() => { clearRules(); registerRules(UnrecognizedRules); }); + + test('extracts argument + partial from message and renders concrete options', () => { + const r = runRules({ + check: 'UnrecognizedRenderPartialArguments', + params: {}, + message: "Unknown argument 'extra' in render tag for partial 'shared/card'.", + }, {}); + expect(r.rule_id).toBe('UnrecognizedRenderPartialArguments.default'); + expect(r.hint_md).toContain('`extra`'); + expect(r.hint_md).toContain('`shared/card`'); + expect(r.hint_md).toContain('`@param`'); + // For project partials, all three options (drop / declare / rename) are valid. + expect(r.hint_md).toContain('Add a matching `@param`'); + }); + + test('module partials disable the "add @param" option', () => { + const r = runRules({ + check: 'UnrecognizedRenderPartialArguments', + params: {}, + message: "Unknown argument 'params' in render tag for partial 'modules/common-styling/toasts'.", + }, {}); + expect(r.hint_md).toContain('module partials are read-only'); + expect(r.fixes[0].description).toContain('Module partials are read-only'); + }); + + test('falls back gracefully when the message can\'t be parsed', () => { + const r = runRules({ + check: 'UnrecognizedRenderPartialArguments', + params: {}, + message: 'unparseable nonsense', + }, {}); + expect(r.rule_id).toBe('UnrecognizedRenderPartialArguments.default'); + expect(r.hint_md).toContain('the unrecognized argument'); + expect(r.hint_md).toContain('the target partial'); + }); +}); + +describe('SchemaProperty rule', () => { + beforeEach(() => { clearRules(); registerRules(SchemaPropertyRules); }); + + const cases = [ + { msg: 'Property name `created_at` conflicts with built-in field. Built-in fields (id, created_at, updated_at, table) are added automatically.', sub: 'builtin_conflict' }, + { msg: 'Duplicate property name `email`. Property names must be unique within a schema.', sub: 'duplicate_name' }, + { msg: 'Property name `2nd_field` must start with a letter, not a digit.', sub: 'invalid_identifier' }, + { msg: 'Property name `myField` should use snake_case (lowercase letters, numbers, underscores).', sub: 'snake_case' }, + { msg: 'Property `avatar`: Unknown upload option `cdn`. Valid options: acl, max_size, content_type.', sub: 'upload_options' }, + { msg: 'properties[2]: Missing required `name` key.', sub: 'missing_field' }, + { msg: 'Property `email`: `required` is not a schema-level concept in platformOS. Validation must be done in mutations/commands.', sub: 'misleading_key' }, + { msg: 'Property `email`: some unfamiliar message', sub: 'default' }, + ]; + + for (const { msg, sub } of cases) { + test(`routes "${msg.slice(0, 40)}..." → SchemaProperty.${sub}`, () => { + const r = runRules({ check: 'pos-supervisor:SchemaProperty', params: {}, message: msg }, {}); + expect(r.rule_id).toBe(`SchemaProperty.${sub}`); + expect(r.fixes).toHaveLength(1); + expect(r.fixes[0].type).toBe('guidance'); + expect(r.see_also.tool).toBe('domain_guide'); + expect(r.see_also.args.domain).toBe('schema'); + }); + } +}); + +describe('SchemaYAML rule', () => { + beforeEach(() => { clearRules(); registerRules(SchemaYAMLRules); }); + + test('attaches stable rule_id + hint with common YAML pitfalls', () => { + const r = runRules({ + check: 'pos-supervisor:SchemaYAML', + params: {}, + message: 'Invalid YAML syntax: expected a single document in the stream', + }, {}); + expect(r.rule_id).toBe('SchemaYAML.default'); + expect(r.hint_md).toContain('Single document only'); + expect(r.hint_md).toContain('Indentation mismatch'); + expect(r.see_also.args.domain).toBe('schema'); + }); +}); + +describe('MissingSlug rule', () => { + beforeEach(() => { clearRules(); registerRules(MissingSlugRules); }); + + test('promotes to stable rule_id and emits guidance only (heuristic owns text_edit)', () => { + const r = runRules({ + check: 'pos-supervisor:MissingSlug', + params: {}, + message: 'Page is missing `slug` in front matter.', + }, {}); + expect(r.rule_id).toBe('MissingSlug.default'); + expect(r.fixes).toHaveLength(1); + expect(r.fixes[0].type).toBe('guidance'); + expect(r.hint_md).toContain('kebab-case'); + expect(r.hint_md).toContain(':param'); + expect(r.see_also.args.domain).toBe('pages'); + }); +}); + +describe('MissingContentForLayout rule', () => { + beforeEach(() => { clearRules(); registerRules(MissingContentRules); }); + + test('promotes to stable rule_id and explains content_for_layout vs yield', () => { + const r = runRules({ + check: 'pos-supervisor:MissingContentForLayout', + params: {}, + message: 'Layout is missing `{{ content_for_layout }}`.', + }, {}); + expect(r.rule_id).toBe('MissingContentForLayout.default'); + expect(r.hint_md).toContain('`{{ content_for_layout }}`'); + expect(r.hint_md).toContain('{% yield'); + expect(r.fixes[0].type).toBe('guidance'); + expect(r.see_also.args.domain).toBe('layouts'); + }); +}); + +describe('ParserBlockingScript rule', () => { + beforeEach(() => { clearRules(); registerRules(ParserBlockingRules); }); + + test('emits decision tree (defer / async / end-of-body)', () => { + const r = runRules({ + check: 'ParserBlockingScript', + params: {}, + message: 'Avoid parser blocking scripts by adding `defer` or `async`', + }, {}); + expect(r.rule_id).toBe('ParserBlockingScript.default'); + expect(r.hint_md).toContain('defer'); + expect(r.hint_md).toContain('async'); + expect(r.fixes[0].description).toContain('defer'); + }); +}); + +describe('TranslationMissingLocaleKey rule', () => { + beforeEach(() => { clearRules(); registerRules(TranslationLocaleRules); }); + + test('extracts locale from message and emits before/after YAML example', () => { + const r = runRules({ + check: 'pos-supervisor:TranslationMissingLocaleKey', + params: {}, + message: "Translation file has no top-level locale key. Top-level keys found: app. Wrap the entire tree in the file's locale (e.g. `en:`) — platformOS indexes translations by locale at the root.", + }, {}); + expect(r.rule_id).toBe('TranslationMissingLocaleKey.default'); + expect(r.hint_md).toMatch(/Wrap the entire tree under `en:`/); + expect(r.hint_md).toContain('# BEFORE'); + expect(r.hint_md).toContain('# AFTER'); + // Extracted locale appears in the generated example. + expect(r.hint_md).toMatch(/^en:/m); + expect(r.see_also.args.domain).toBe('translations'); + }); + + test('handles non-en locales (de, pt-BR)', () => { + const r = runRules({ + check: 'pos-supervisor:TranslationMissingLocaleKey', + params: {}, + message: "Translation file has no top-level locale key. Top-level keys found: app. Wrap the entire tree in the file's locale (e.g. `pt-BR:`) — platformOS indexes translations by locale at the root.", + }, {}); + expect(r.hint_md).toMatch(/Wrap the entire tree under `pt-BR:`/); + expect(r.hint_md).toMatch(/^pt-BR:/m); + }); + + test('falls back to `en` when locale hint missing from message', () => { + const r = runRules({ + check: 'pos-supervisor:TranslationMissingLocaleKey', + params: {}, + message: 'Translation file has no top-level locale key.', + }, {}); + expect(r.hint_md).toMatch(/Wrap the entire tree under `en:`/); + }); +}); diff --git a/tests/unit/rules/Tier3RulesPhase2.test.js b/tests/unit/rules/Tier3RulesPhase2.test.js new file mode 100644 index 0000000..b44f80f --- /dev/null +++ b/tests/unit/rules/Tier3RulesPhase2.test.js @@ -0,0 +1,204 @@ +// Tier 3 phase 2 — Levenshtein + structural rule modules: +// MissingAsset, OrphanedPartial, MissingPage, LiquidHTMLSyntaxError. + +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { buildFactGraph } from '../../../src/core/project-fact-graph.js'; + +import { rules as MissingAssetRules } from '../../../src/core/rules/MissingAsset.js'; +import { rules as OrphanedPartialRules } from '../../../src/core/rules/OrphanedPartial.js'; +import { rules as MissingPageRules } from '../../../src/core/rules/MissingPage.js'; +import { rules as LiquidHTMLSyntaxErrorRules } from '../../../src/core/rules/LiquidHTMLSyntaxError.js'; + +describe('MissingAsset rule', () => { + const graph = buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, + assets: ['images/logo.png', 'styles/main.css', 'styles/main.scss', 'scripts/app.js'], + }); + const facts = { graph }; + + beforeEach(() => { clearRules(); registerRules(MissingAssetRules); }); + + test('subdir_prefix: bare filename matching a known-subdir asset → fix the reference', () => { + const r = runRules({ check: 'MissingAsset', message: "'logo.png' does not exist" }, facts); + expect(r.rule_id).toBe('MissingAsset.missing_subdir_prefix'); + expect(r.hint_md).toContain('images/logo.png'); + expect(r.fixes[0].description).toContain('images/logo.png'); + expect(r.confidence).toBeGreaterThanOrEqual(0.85); + }); + + test('suggest_nearest: typo in subdir asset → Levenshtein candidates', () => { + const r = runRules({ check: 'MissingAsset', message: "'styles/maain.css' does not exist" }, facts); + expect(r.rule_id).toBe('MissingAsset.suggest_nearest'); + expect(r.hint_md).toContain('styles/main.css'); + }); + + test('create_file: no near match → propose creation, lower confidence', () => { + const r = runRules({ check: 'MissingAsset', message: "'foo/bar.css' does not exist" }, facts); + expect(r.rule_id).toBe('MissingAsset.create_file'); + expect(r.confidence).toBeLessThanOrEqual(0.7); + }); + + test('subdir_prefix only fires for known asset subdirs (avoids false matches)', () => { + const stranger = buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, + assets: ['vendor/data/logo.png'], // NOT under known subdir + }); + clearRules(); registerRules(MissingAssetRules); + const r = runRules({ check: 'MissingAsset', message: "'logo.png' does not exist" }, { graph: stranger }); + // Should NOT match subdir_prefix — `vendor` is not in KNOWN_ASSET_SUBDIRS. + // Falls through to suggest_nearest (or create_file if no Levenshtein match). + expect(r.rule_id).not.toBe('MissingAsset.missing_subdir_prefix'); + }); +}); + +describe('OrphanedPartial rule', () => { + beforeEach(() => { clearRules(); registerRules(OrphanedPartialRules); }); + + test('partial with zero callers → propose delete_file + guidance', () => { + const graph = buildFactGraph({ + pages: {}, partials: { + 'foo/orphan': { path: 'app/views/partials/foo/orphan.liquid', params: [], renders: [], render_calls: [], function_calls: [], rendered_by: [] }, + }, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); + const r = runRules({ + check: 'OrphanedPartial', + file: 'app/views/partials/foo/orphan.liquid', + message: 'This partial is not referenced by any other files', + }, { graph }); + expect(r.rule_id).toBe('OrphanedPartial.default'); + expect(r.fixes.some(f => f.type === 'delete_file')).toBe(true); + expect(r.fixes.some(f => f.type === 'guidance')).toBe(true); + expect(r.hint_md).toContain('Work in progress'); + expect(r.hint_md).toContain('pending_files'); + }); + + test('layout with no callers → softer guidance, no delete_file', () => { + const graph = buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, + layouts: { + 'app/views/layouts/unused.liquid': { path: 'app/views/layouts/unused.liquid', renders: [], render_calls: [], function_calls: [] }, + }, + translations: {}, assets: [], + }); + const r = runRules({ + check: 'OrphanedPartial', + file: 'app/views/layouts/unused.liquid', + message: 'This partial is not referenced by any other files', + }, { graph }); + expect(r.rule_id).toBe('OrphanedPartial.default'); + expect(r.fixes.some(f => f.type === 'delete_file')).toBe(false); + expect(r.hint_md).toContain('layout'); + }); + + test('falls back gracefully without diag.file', () => { + const r = runRules({ + check: 'OrphanedPartial', + message: 'This partial is not referenced by any other files', + }, { graph: buildFactGraph({ pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [] }) }); + expect(r.rule_id).toBe('OrphanedPartial.default'); + // Without a path, no delete_file proposal — too dangerous. + expect(r.fixes.some(f => f.type === 'delete_file')).toBe(false); + }); +}); + +describe('MissingPage rule', () => { + const graph = buildFactGraph({ + pages: { + 'idx': { path: 'app/views/pages/index.liquid', slug: '', method: 'get', renders: [] }, + 'notes:get': { path: 'app/views/pages/notes/index.html.liquid', slug: 'notes', method: 'get', renders: [] }, + 'dashboard:get': { path: 'app/views/pages/dashboard.liquid', slug: 'dashboard', method: 'get', renders: [] }, + }, + partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); + + beforeEach(() => { clearRules(); registerRules(MissingPageRules); }); + + test('typo: close to existing slug → suggest rename', () => { + const r = runRules({ + check: 'MissingPage', + message: "No page found for route '/noets' (GET)", + }, { graph }); + expect(r.rule_id).toBe('MissingPage.typo'); + expect(r.hint_md).toContain('/notes'); + }); + + test('default: no near match → three-option decision tree + create_file', () => { + const r = runRules({ + check: 'MissingPage', + message: "No page found for route '/profile' (GET)", + }, { graph }); + expect(r.rule_id).toBe('MissingPage.default'); + expect(r.hint_md).toContain('Typo in the reference'); + expect(r.hint_md).toContain('New page'); + expect(r.hint_md).toContain('Method mismatch'); + expect(r.fixes[0].type).toBe('create_file'); + expect(r.fixes[0].path).toBe('app/views/pages/profile.liquid'); + }); + + test('root route → suggests omitting slug, points at index.liquid', () => { + const r = runRules({ + check: 'MissingPage', + message: "No page found for route '/' (GET)", + }, { graph: buildFactGraph({ pages:{}, partials:{}, commands:{}, queries:{}, graphql:{}, schema:{}, layouts:{}, translations:{}, assets:[] }) }); + expect(r.rule_id).toBe('MissingPage.default'); + expect(r.fixes[0].path).toBe('app/views/pages/index.liquid'); + expect(r.hint_md).toContain('omit `slug:`'); + }); + + test('extracts method from message', () => { + const r = runRules({ + check: 'MissingPage', + message: "No page found for route '/api/sync' (POST)", + }, { graph }); + expect(r.hint_md).toContain('(POST)'); + }); +}); + +describe('LiquidHTMLSyntaxError rule', () => { + beforeEach(() => { clearRules(); registerRules(LiquidHTMLSyntaxErrorRules); }); + + test('unknown_tag fires on "Unknown tag" message + suggests via tagsIndex', () => { + const tagsIndex = { + platformOSTags: () => [ + { name: 'assign' }, { name: 'render' }, { name: 'function' }, { name: 'graphql' }, { name: 'if' }, { name: 'for' }, + ], + }; + const r = runRules({ + check: 'LiquidHTMLSyntaxError', + message: "Unknown tag 'assigns'", + }, { tagsIndex }); + expect(r.rule_id).toBe('LiquidHTMLSyntaxError.unknown_tag'); + expect(r.hint_md).toContain('assign'); + }); + + test('unknown_tag works without tagsIndex (no suggestion, still attributes)', () => { + const r = runRules({ + check: 'LiquidHTMLSyntaxError', + message: "Unknown tag 'foo'", + }, {}); + expect(r.rule_id).toBe('LiquidHTMLSyntaxError.unknown_tag'); + expect(r.hint_md).toContain('foo'); + }); + + test('for_loop_args fires when filter pipeline appears in for loop header', () => { + const r = runRules({ + check: 'LiquidHTMLSyntaxError', + message: "Arguments must be provided in the format `for in `. Invalid/Unknown arguments: |, t", + }, {}); + expect(r.rule_id).toBe('LiquidHTMLSyntaxError.for_loop_args'); + expect(r.hint_md).toContain('assign items'); + // The fix description cross-references the translation-array sibling rule + // (the most common origin of `| t` inside a for header). + expect(r.fixes[0].description).toContain('TranslationKeyExists.array_index_misuse'); + }); + + test('default fallback for unknown shapes', () => { + const r = runRules({ + check: 'LiquidHTMLSyntaxError', + message: 'something obscure happened', + }, {}); + expect(r.rule_id).toBe('LiquidHTMLSyntaxError.default'); + expect(r.confidence).toBeLessThanOrEqual(0.6); + }); +}); diff --git a/tests/unit/rules/Tier3RulesPhase3.test.js b/tests/unit/rules/Tier3RulesPhase3.test.js new file mode 100644 index 0000000..ab242f5 --- /dev/null +++ b/tests/unit/rules/Tier3RulesPhase3.test.js @@ -0,0 +1,358 @@ +// Tier-3 phase 3 — high-volume bucket-B promotions: +// • PartialCallArguments (28 emits in DEMO; 4 subrules) +// • GraphQLVariablesCheck (3 emits; signature block via graph) +// • UnusedDocParam (11 emits; caller-aware confidence) +// +// Also covers the diagnostic-record extractors that feed these rules. + +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { extractParams } from '../../../src/core/diagnostic-record.js'; +import { buildFactGraph } from '../../../src/core/project-fact-graph.js'; + +import { rules as PartialCallArgumentsRules } from '../../../src/core/rules/PartialCallArguments.js'; +import { rules as GraphQLVariablesCheckRules } from '../../../src/core/rules/GraphQLVariablesCheck.js'; +import { rules as UnusedDocParamRules } from '../../../src/core/rules/UnusedDocParam.js'; + +describe('extractParams — PartialCallArguments / GraphQLVariablesCheck / UnusedDocParam', () => { + test('PartialCallArguments — required, function call', () => { + expect(extractParams('PartialCallArguments', 'Required parameter key must be passed to function call')) + .toEqual({ param_name: 'key', direction: 'required', call_kind: 'function', is_function_call: 'true' }); + }); + test('PartialCallArguments — required, render call', () => { + expect(extractParams('PartialCallArguments', 'Required parameter success must be passed to render call')) + .toEqual({ param_name: 'success', direction: 'required', call_kind: 'render', is_function_call: 'false' }); + }); + test('PartialCallArguments — unknown, render call', () => { + expect(extractParams('PartialCallArguments', 'Unknown parameter params passed to render call')) + .toEqual({ param_name: 'params', direction: 'unknown', call_kind: 'render', is_function_call: 'false' }); + }); + test('PartialCallArguments — extractor returns {} on unknown shape', () => { + expect(extractParams('PartialCallArguments', 'something brand new')).toEqual({}); + }); + test('GraphQLVariablesCheck — required', () => { + expect(extractParams('GraphQLVariablesCheck', 'Required parameter name must be passed to GraphQL call')) + .toEqual({ param_name: 'name', direction: 'required', call_kind: 'graphql' }); + }); + test('GraphQLVariablesCheck — unknown', () => { + expect(extractParams('GraphQLVariablesCheck', 'Unknown parameter foo passed to GraphQL call')) + .toEqual({ param_name: 'foo', direction: 'unknown', call_kind: 'graphql' }); + }); + test('UnusedDocParam — extractor', () => { + expect(extractParams('UnusedDocParam', "The parameter 'title' is defined but not used in this file.")) + .toEqual({ param_name: 'title' }); + expect(extractParams('UnusedDocParam', 'unparseable')) + .toEqual({}); + }); +}); + +describe('PartialCallArguments rule', () => { + beforeEach(() => { clearRules(); registerRules(PartialCallArgumentsRules); }); + + function run(message) { + const params = extractParams('PartialCallArguments', message); + return runRules({ check: 'PartialCallArguments', params, message }, {}); + } + + test('required + render → required_render with `success: success` example', () => { + const r = run('Required parameter success must be passed to render call'); + expect(r.rule_id).toBe('PartialCallArguments.required_render'); + expect(r.hint_md).toContain('forward caller'); + expect(r.hint_md).toMatch(/render '[^']+', success: success/); + expect(r.confidence).toBe(0.7); + expect(r.see_also.args.domain).toBe('partials'); + }); + + test('required + function → required_function with `function r = ...` example', () => { + const r = run('Required parameter key must be passed to function call'); + expect(r.rule_id).toBe('PartialCallArguments.required_function'); + expect(r.hint_md).toMatch(/function r = '[^']+', key: key/); + expect(r.see_also.args.domain).toBe('commands'); + }); + + test('unknown + render → unknown_render with three-option (drop / declare / rename)', () => { + const r = run('Unknown parameter params passed to render call'); + expect(r.rule_id).toBe('PartialCallArguments.unknown_render'); + expect(r.hint_md).toContain('Drop'); + expect(r.hint_md).toContain('Declare'); + expect(r.hint_md).toContain('Rename'); + expect(r.fixes[0].description).toMatch(/[Mm]odule-owned/); + }); + + test('unknown + function → unknown_function', () => { + const r = run('Unknown parameter from passed to function call'); + expect(r.rule_id).toBe('PartialCallArguments.unknown_function'); + }); + + test('cross-references the sibling Missing*Arguments check', () => { + const r = run('Required parameter success must be passed to render call'); + expect(r.hint_md).toContain('MissingRenderPartialArguments'); + }); + + test('default fallback when extractor produces no params', () => { + const r = run('Some new diagnostic shape'); + expect(r.rule_id).toBe('PartialCallArguments.default'); + expect(r.confidence).toBeLessThanOrEqual(0.5); + }); +}); + +describe('GraphQLVariablesCheck rule', () => { + beforeEach(() => { clearRules(); registerRules(GraphQLVariablesCheckRules); }); + + // Minimal fixture: one page that calls a graphql operation with two + // declared variables. Graph maps page path → graphql_calls; graphql + // node carries the args list. + const graph = buildFactGraph({ + pages: { + 'idx': { + path: 'app/views/pages/contact.liquid', + slug: 'contact', + method: 'post', + renders: [], + function_calls: [], + graphql_calls: [{ variable: 'r', queryName: 'contact_messages/create' }], + }, + }, + partials: {}, commands: {}, queries: {}, + graphql: { + 'contact_messages/create': { + operation: 'mutation', + name: 'create', + args: [{ name: 'name', type: 'String!' }, { name: 'email', type: 'String!' }], + table: 'contact', + }, + }, + schema: {}, layouts: {}, translations: {}, assets: [], + }); + + test('required → ships canonical examples + signature block from graph', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Required parameter name must be passed to GraphQL call'), + message: 'Required parameter name must be passed to GraphQL call', + file: 'app/views/pages/contact.liquid', + }, { graph }); + expect(r.rule_id).toBe('GraphQLVariablesCheck.required'); + expect(r.hint_md).toContain('contact_messages/create'); + expect(r.hint_md).toContain('$name: String!'); + expect(r.hint_md).toContain('$email: String!'); + expect(r.fixes[0].description).toContain('app/graphql/contact_messages/create.graphql'); + }); + + test('unknown → 3-option fix, with signature block when graph has the call', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Unknown parameter foo passed to GraphQL call'), + message: 'Unknown parameter foo passed to GraphQL call', + file: 'app/views/pages/contact.liquid', + }, { graph }); + expect(r.rule_id).toBe('GraphQLVariablesCheck.unknown'); + expect(r.hint_md).toContain('Drop'); + expect(r.hint_md).toContain('Declare'); + expect(r.hint_md).toContain('Rename'); + expect(r.hint_md).toContain('contact_messages/create'); + }); + + test('signature block omitted when caller file is unknown to graph', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Required parameter name must be passed to GraphQL call'), + message: 'Required parameter name must be passed to GraphQL call', + file: 'app/views/pages/orphan.liquid', + }, { graph }); + expect(r.rule_id).toBe('GraphQLVariablesCheck.required'); + expect(r.hint_md).not.toContain('GraphQL operation(s) called'); + }); + + test('default fallback', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'something obscure'), + message: 'something obscure', + }, {}); + expect(r.rule_id).toBe('GraphQLVariablesCheck.default'); + }); + + // Repro for the DEMO 2026-04-27 regression spiral. When the project + // graph reports the file's graphql call with source_kind=liquid_multiline_truncated, + // the parser_blind_spot sub-rule must fire BEFORE .required and steer the + // agent at the syntactic root cause. + describe('parser_blind_spot — multi-line truncation', () => { + const truncatedGraph = buildFactGraph({ + pages: {}, + partials: {}, + commands: { + 'app/lib/commands/contacts/create.liquid': { + path: 'app/lib/commands/contacts/create.liquid', + renders: [], + function_calls: [], + graphql_calls: [{ + variable: 'result', + queryName: 'contacts/create', + args: [], + source_kind: 'liquid_multiline_truncated', + }], + }, + }, + queries: {}, + graphql: { + 'contacts/create': { + operation: 'mutation', + name: 'create', + args: [ + { name: 'name', type: 'String!' }, + { name: 'email', type: 'String!' }, + ], + table: 'contact', + }, + }, + schema: {}, layouts: {}, translations: {}, assets: [], + }); + + test('fires before .required when graph flags the call truncated', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Required parameter name must be passed to GraphQL call'), + message: 'Required parameter name must be passed to GraphQL call', + file: 'app/lib/commands/contacts/create.liquid', + }, { graph: truncatedGraph }); + expect(r.rule_id).toBe('GraphQLVariablesCheck.parser_blind_spot'); + expect(r.hint_md).toContain('parser cannot see it'); + expect(r.hint_md).toContain('Fix the syntax'); + expect(r.hint_md).toContain('contacts/create'); + expect(r.confidence).toBe(0.95); + }); + + test('does NOT fire for direction=unknown — only required suffers from this blind spot', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Unknown parameter foo passed to GraphQL call'), + message: 'Unknown parameter foo passed to GraphQL call', + file: 'app/lib/commands/contacts/create.liquid', + }, { graph: truncatedGraph }); + expect(r.rule_id).toBe('GraphQLVariablesCheck.unknown'); + }); + + test('falls through to .required when the call is NOT truncated', () => { + const okGraph = buildFactGraph({ + pages: {}, + partials: {}, + commands: { + 'app/lib/commands/contacts/create.liquid': { + path: 'app/lib/commands/contacts/create.liquid', + renders: [], + function_calls: [], + graphql_calls: [{ + variable: 'result', + queryName: 'contacts/create', + args: ['name', 'email'], + source_kind: 'tag', + }], + }, + }, + queries: {}, + graphql: { + 'contacts/create': { + operation: 'mutation', name: 'create', + args: [{ name: 'name', type: 'String!' }, { name: 'email', type: 'String!' }], + table: 'contact', + }, + }, + schema: {}, layouts: {}, translations: {}, assets: [], + }); + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Required parameter name must be passed to GraphQL call'), + message: 'Required parameter name must be passed to GraphQL call', + file: 'app/lib/commands/contacts/create.liquid', + }, { graph: okGraph }); + expect(r.rule_id).toBe('GraphQLVariablesCheck.required'); + }); + + test('falls through to .required when the file is not in the graph', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Required parameter name must be passed to GraphQL call'), + message: 'Required parameter name must be passed to GraphQL call', + file: 'app/views/pages/orphan.liquid', + }, { graph: truncatedGraph }); + expect(r.rule_id).toBe('GraphQLVariablesCheck.required'); + }); + + test('safe when no graph is available (degrades to .required)', () => { + const r = runRules({ + check: 'GraphQLVariablesCheck', + params: extractParams('GraphQLVariablesCheck', 'Required parameter name must be passed to GraphQL call'), + message: 'Required parameter name must be passed to GraphQL call', + file: 'app/lib/commands/contacts/create.liquid', + }, {}); + expect(r.rule_id).toBe('GraphQLVariablesCheck.required'); + }); + }); +}); + +describe('UnusedDocParam rule', () => { + beforeEach(() => { clearRules(); registerRules(UnusedDocParamRules); }); + + test('lone partial (zero callers in graph) → safer to remove, higher confidence', () => { + const graph = buildFactGraph({ + pages: {}, partials: { + 'shared/orphan': { path: 'app/views/partials/shared/orphan.liquid', params: ['title'], renders: [], function_calls: [], rendered_by: [] }, + }, + commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); + const r = runRules({ + check: 'UnusedDocParam', + params: extractParams('UnusedDocParam', "The parameter 'title' is defined but not used in this file."), + message: "The parameter 'title' is defined but not used in this file.", + file: 'app/views/partials/shared/orphan.liquid', + }, { graph }); + expect(r.rule_id).toBe('UnusedDocParam.default'); + expect(r.confidence).toBe(0.8); + expect(r.fixes[0].description).toMatch(/option B \(remove `@param title`[^)]*\) is safe/); + }); + + test('partial with callers → lower confidence, warns about contract change', () => { + const graph = buildFactGraph({ + pages: { + 'idx': { path: 'app/views/pages/index.liquid', renders: ['shared/card'], render_calls: [{ partial: 'shared/card', args: ['title'] }], function_calls: [] }, + }, + partials: { + 'shared/card': { path: 'app/views/partials/shared/card.liquid', params: ['title'], renders: [], function_calls: [], rendered_by: [] }, + }, + commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); + const r = runRules({ + check: 'UnusedDocParam', + params: extractParams('UnusedDocParam', "The parameter 'title' is defined but not used in this file."), + message: "The parameter 'title' is defined but not used in this file.", + file: 'app/views/partials/shared/card.liquid', + }, { graph }); + expect(r.rule_id).toBe('UnusedDocParam.default'); + expect(r.confidence).toBe(0.65); + expect(r.hint_md).toContain('caller(s) reference this file'); + expect(r.fixes[0].description).toContain('caller(s) reference this file'); + }); + + test('no graph / file → degraded but functional', () => { + const r = runRules({ + check: 'UnusedDocParam', + params: extractParams('UnusedDocParam', "The parameter 'foo' is defined but not used in this file."), + message: "The parameter 'foo' is defined but not used in this file.", + }, {}); + expect(r.rule_id).toBe('UnusedDocParam.default'); + expect(r.hint_md).toContain('Caller count unknown'); + expect(r.hint_md).toContain('platformos_references'); + }); + + test('hint references the pipeline pre-suppression', () => { + const r = runRules({ + check: 'UnusedDocParam', + params: extractParams('UnusedDocParam', "The parameter 'foo' is defined but not used in this file."), + message: "The parameter 'foo' is defined but not used in this file.", + }, {}); + // The diagnosis emphasises that named-arg use in this file is already + // suppressed upstream — surviving emits are real dead declarations. + expect(r.hint_md).toContain('pipeline already suppresses'); + }); +}); diff --git a/tests/unit/rules/TranslationKeyExists.test.js b/tests/unit/rules/TranslationKeyExists.test.js new file mode 100644 index 0000000..1b67450 --- /dev/null +++ b/tests/unit/rules/TranslationKeyExists.test.js @@ -0,0 +1,320 @@ +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/TranslationKeyExists.js'; +import { buildFactGraph } from '../../../src/core/project-fact-graph.js'; + +const graph = buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, + translations: { en: { 'app.title': 'Blog', 'app.subtitle': 'Posts', 'common.save': 'Save', 'common.cancel': 'Cancel' } }, + assets: [], +}); +const facts = { graph }; + +beforeEach(() => { clearRules(); registerRules(rules); }); + +describe('TranslationKeyExists.suggest_nearest', () => { + test('suggests similar keys', () => { + const diag = { check: 'TranslationKeyExists', params: { key: 'app.titl' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.suggest_nearest'); + expect(result.hint_md).toContain('app.title'); + }); + + test('suggests from common namespace', () => { + const diag = { check: 'TranslationKeyExists', params: { key: 'common.sav' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.suggest_nearest'); + expect(result.hint_md).toContain('common.save'); + }); +}); + +describe('TranslationKeyExists.create_key', () => { + test('suggests creating new key with YAML snippet', () => { + const diag = { check: 'TranslationKeyExists', params: { key: 'products.new.heading' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.create_key'); + expect(result.hint_md).toContain('products'); + expect(result.hint_md).toContain('heading'); + expect(result.hint_md).toContain('en.yml'); + }); + + test('handles single-segment key', () => { + const diag = { check: 'TranslationKeyExists', params: { key: 'greeting' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.create_key'); + expect(result.hint_md).toContain('greeting'); + }); +}); + +describe('TranslationKeyExists.array_index_misuse', () => { + test('fires on `key[0]` and provides iteration guidance instead of nearest', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'landing.problem.items[0]' }, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + // Hint references the canonical iteration pattern, NOT a "did you mean" suggestion. + expect(result.hint_md).toMatch(/assign items/); + expect(result.hint_md).toMatch(/landing\.problem\.items/); + // The arrayKey reference must NOT carry the [0] suffix. + expect(result.hint_md).not.toMatch(/landing\.problem\.items\[0\]['"]\s*\| t/); + expect(result.fixes).toHaveLength(1); + expect(result.fixes[0].type).toBe('guidance'); + expect(result.fixes[0].description).toMatch(/\{% for item in items %\}/); + expect(result.confidence).toBe(0.9); + }); + + test('also catches multi-segment indices (`items[12]`)', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'landing.problem.items[12]' }, + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + }); + + test('suggest_nearest does NOT fire for indexed keys (would be misleading)', () => { + // Even with a Levenshtein-close parent key in the graph, the array-index + // rule wins by priority. The suggest_nearest path is gated by an explicit + // check so it never produces "did you mean en.parent.items". + const indexedFacts = { + graph: buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, + translations: { en: { 'landing.problem.items': ['a', 'b'] } }, + assets: [], + }), + }; + const diag = { + check: 'TranslationKeyExists', + params: { key: 'landing.problem.items[0]' }, + }; + const result = runRules(diag, indexedFacts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + expect(result.rule_id).not.toBe('TranslationKeyExists.suggest_nearest'); + }); + + test('create_key does NOT fire for indexed keys (would propose nonsense YAML)', () => { + // Empty translations graph forces create_key to be the only otherwise-eligible rule. + // Array-index rule must still win. + const emptyFacts = { + graph: buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, + translations: {}, assets: [], + }), + }; + const diag = { + check: 'TranslationKeyExists', + params: { key: 'landing.problem.items[0]' }, + }; + const result = runRules(diag, emptyFacts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + }); +}); + +describe('TranslationKeyExists — edge cases', () => { + test('falls through to .default when key param is missing', () => { + const diag = { check: 'TranslationKeyExists', params: {} }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('TranslationKeyExists.default'); + }); +}); + +describe('TranslationKeyExists.default catch-all', () => { + test('does NOT preempt .array_index_misuse', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'app.title[0]' }, + message: "'app.title[0]' does not have a matching translation entry", + }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + }); + + test('does NOT preempt .suggest_nearest', () => { + const diag = { check: 'TranslationKeyExists', params: { key: 'app.titl' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.suggest_nearest'); + }); + + test('does NOT preempt .create_key for a brand-new key with no near matches', () => { + const diag = { check: 'TranslationKeyExists', params: { key: 'a.completely.disjoint.brand_new.key' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('TranslationKeyExists.create_key'); + }); + + test('fires when extraction failed entirely', () => { + const diag = { check: 'TranslationKeyExists' }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('TranslationKeyExists.default'); + expect(result.confidence).toBeLessThanOrEqual(0.5); + }); + + test('hint warns against locale-prefix typos and points at app/translations/', () => { + const diag = { check: 'TranslationKeyExists' }; + const result = runRules(diag, facts); + expect(result.hint_md).toContain('app/translations/'); + expect(result.hint_md).toContain('| t'); + expect(result.hint_md).toContain('locale'); + }); +}); + +// Realistic translations shape — what `flattenYaml` actually emits when the +// YAML root is `en:` (the platformOS-required wrapper). Every key carries +// the locale prefix. +const realisticGraph = buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, + translations: { + en: { + 'en.landing.problem.items': ['a', 'b'], + 'en.landing.problem.title': 'Problem', + 'en.landing.proof.title': 'Proof', + 'en.app.user.title': 'User', + 'en.app.user.name': 'Name', + }, + }, + assets: [], +}); +const realisticFacts = { graph: realisticGraph }; + +describe('TranslationKeyExists.suggest_nearest — locale-prefix correctness', () => { + test('hint emits bare keys (no `en.` prefix) for graph keys built from realistic YAML', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'app.usr.title' }, + message: "'app.usr.title' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.suggest_nearest'); + // Suggested key must NOT carry the `en.` prefix — Liquid's `| t` filter + // re-prepends the locale, so suggesting `en.app.user.title` makes the + // agent's call resolve to `en.en.app.user.title` and fail again. + expect(result.hint_md).toContain('app.user.title'); + expect(result.hint_md).not.toMatch(/`en\.app\.user\.title`/); + expect(result.fixes[0].description).not.toMatch(/`en\.app\.user\.title`/); + }); + + test('hint warns explicitly against including the locale prefix', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'app.usr.title' }, + message: "'app.usr.title' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.hint_md).toMatch(/do NOT include `en\.`/i); + }); + + test('agent supplied an `en.`-prefixed key — rule strips it before matching', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'en.app.usr.title' }, + message: "'en.app.usr.title' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.suggest_nearest'); + expect(result.hint_md).toContain('app.user.title'); + }); + + test('brand-new key with no close match falls through to create_key (stricter threshold)', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'app.brand_new_feature.label' }, + message: "'app.brand_new_feature.label' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.create_key'); + }); + + test('one-character typo on a real key still suggests', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'app.user.namee' }, + message: "'app.user.namee' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.suggest_nearest'); + expect(result.hint_md).toContain('app.user.name'); + }); +}); + +describe('TranslationKeyExists.create_key — locale-prefix correctness', () => { + test('agent-supplied `en.` prefix is stripped before YAML emission', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'en.products.heading' }, + message: "'en.products.heading' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.create_key'); + // The YAML snippet nests under `products:` (NOT `en: products:`) because + // the file already has the `en:` root and prepending again would create + // `en.en.products.heading` at lookup time. + expect(result.hint_md).toMatch(/^products:/m); + expect(result.hint_md).not.toMatch(/^en:\s*\n\s*products:/m); + }); + + test('clarifies the YAML must nest under the existing `en:` root', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'app.greeting' }, + message: "'app.greeting' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.fixes[0].description).toMatch(/nested under the existing `en:` root/); + }); +}); + +describe('TranslationKeyExists.array_index_misuse — defensive gate', () => { + test('hint suggests bare arrayKey even when agent prefixed with `en.`', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'en.landing.problem.items[2]' }, + message: "'en.landing.problem.items[2]' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + // The `assign items = '...'` snippet must NOT include `en.` — the agent + // would otherwise write `'en.landing.problem.items' | t` and re-trigger + // the prefix double-up. + expect(result.hint_md).toMatch(/assign items = 'landing\.problem\.items'/); + expect(result.hint_md).not.toMatch(/assign items = 'en\.landing\.problem\.items'/); + }); + + test('raw-message gate: catches `[N]` even when params.key extraction loses it', () => { + // Belt-and-suspenders: if the extractor ever drops the bracket from + // params.key (LSP shape change, encoding bug), the raw-message regex + // still routes to array_index_misuse instead of letting suggest_nearest + // emit a misleading parent-key suggestion. + const diag = { + check: 'TranslationKeyExists', + params: { key: 'landing.problem.items' }, // no [N] in params + message: "'landing.problem.items[3]' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + }); + + test('suggest_nearest is gated by raw-message regex too', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'landing.problem.items' }, + message: "'landing.problem.items[3]' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + // Even though params.key has no [N] and is Levenshtein-close to a real key, + // the rule must defer to array_index_misuse via the raw-message gate. + expect(result.rule_id).not.toBe('TranslationKeyExists.suggest_nearest'); + }); + + test('create_key is gated by raw-message regex too', () => { + const diag = { + check: 'TranslationKeyExists', + params: { key: 'something_unrelated' }, + message: "'something_unrelated[0]' does not have a matching translation entry", + }; + const result = runRules(diag, realisticFacts); + expect(result.rule_id).toBe('TranslationKeyExists.array_index_misuse'); + }); +}); diff --git a/tests/unit/rules/UndefinedObject.test.js b/tests/unit/rules/UndefinedObject.test.js new file mode 100644 index 0000000..0c350af --- /dev/null +++ b/tests/unit/rules/UndefinedObject.test.js @@ -0,0 +1,142 @@ +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/UndefinedObject.js'; +import { buildFactGraph } from '../../../src/core/project-fact-graph.js'; + +const graph = buildFactGraph({ pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [] }); +const facts = { graph }; + +beforeEach(() => { clearRules(); registerRules(rules); }); + +describe('UndefinedObject.shopify_object', () => { + test('fires for Shopify theme objects', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'product' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.shopify_object'); + expect(result.confidence).toBe(0.95); + expect(result.hint_md).toContain('Shopify'); + }); + + test('fires for cart', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'cart' }, file: 'app/views/pages/cart.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.shopify_object'); + }); + + test('does not fire for non-Shopify variables', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'my_var' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).not.toBe('UndefinedObject.shopify_object'); + }); +}); + +describe('UndefinedObject.context_prefix', () => { + test('fires for bare context properties in pages', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'params' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.context_prefix'); + expect(result.hint_md).toContain('context.params'); + }); + + test('fires for session in pages', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'session' }, file: 'app/views/pages/login.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.context_prefix'); + }); + + test('does not fire in partials', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'params' }, file: 'app/views/partials/header.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).not.toBe('UndefinedObject.context_prefix'); + }); + + test('does not fire for non-context variables', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'my_var' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).not.toBe('UndefinedObject.context_prefix'); + }); +}); + +describe('UndefinedObject.declare_param', () => { + test('fires for undefined var in partials', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'item' }, file: 'app/views/partials/card.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.declare_param'); + expect(result.hint_md).toContain('@param'); + expect(result.hint_md).toContain('item'); + }); + + test('fires for undefined var in commands', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'object' }, file: 'app/lib/commands/blog_posts/create.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.declare_param'); + expect(result.hint_md).toContain('command'); + }); + + test('fires for undefined var in queries', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'id' }, file: 'app/lib/queries/blog_posts/find.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.declare_param'); + expect(result.hint_md).toContain('query'); + }); +}); + +describe('UndefinedObject.generic', () => { + test('fallback for unknown var in pages', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'xyz' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.generic'); + expect(result.confidence).toBe(0.5); + }); +}); + +describe('UndefinedObject — edge cases', () => { + test('falls through to .default when variable param is missing', () => { + const diag = { check: 'UndefinedObject', params: {} }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UndefinedObject.default'); + }); +}); + +describe('UndefinedObject.default catch-all', () => { + test('does NOT preempt .shopify_object', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'product' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.shopify_object'); + }); + + test('does NOT preempt .context_prefix', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'params' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.context_prefix'); + }); + + test('does NOT preempt .declare_param in partials', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'props' }, file: 'app/views/partials/header.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.declare_param'); + }); + + test('does NOT preempt .generic when variable is extracted', () => { + const diag = { check: 'UndefinedObject', params: { variable: 'xyz' }, file: 'app/views/pages/index.html.liquid' }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UndefinedObject.generic'); + }); + + test('fires when extraction failed entirely (no params, no file)', () => { + const diag = { check: 'UndefinedObject' }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UndefinedObject.default'); + expect(result.confidence).toBeLessThan(0.5); + }); + + test('hint covers the three canonical resolutions (page / partial / local)', () => { + const diag = { check: 'UndefinedObject', params: {} }; + const result = runRules(diag, facts); + expect(result.hint_md).toContain('context.'); + expect(result.hint_md).toContain('@param'); + expect(result.hint_md).toContain('assign'); + }); +}); diff --git a/tests/unit/rules/UnknownFilter.test.js b/tests/unit/rules/UnknownFilter.test.js new file mode 100644 index 0000000..13dcf61 --- /dev/null +++ b/tests/unit/rules/UnknownFilter.test.js @@ -0,0 +1,131 @@ +import { describe, test, expect, beforeEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/UnknownFilter.js'; +import { buildFactGraph } from '../../../src/core/project-fact-graph.js'; + +const graph = buildFactGraph({ pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [] }); + +const mockTagsIndex = { + isTag: (name) => ['render', 'graphql', 'function', 'if', 'for'].includes(name), +}; + +const mockFiltersIndex = { + loaded: true, + lookup: (name) => name === 'downcase' ? { name: 'downcase', syntax: '{{ string | downcase }}', summary: 'Lowercase' } : null, + closestMatch: (name) => name === 'doncase' ? { name: 'downcase', syntax: '{{ string | downcase }}', summary: 'Lowercase' } : null, +}; + +beforeEach(() => { clearRules(); registerRules(rules); }); + +describe('UnknownFilter.tag_confusion', () => { + test('fires when filter name is actually a tag', () => { + const facts = { graph, tagsIndex: mockTagsIndex, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'render' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.tag_confusion'); + expect(result.hint_md).toContain('tag, not a filter'); + expect(result.confidence).toBe(0.95); + }); +}); + +describe('UnknownFilter.shopify_filter', () => { + test('fires for Shopify-specific filters', () => { + const facts = { graph, tagsIndex: { isTag: () => false }, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'money' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.shopify_filter'); + expect(result.hint_md).toContain('Shopify'); + }); + + test('fires for img_url', () => { + const facts = { graph, tagsIndex: { isTag: () => false }, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'img_url' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.shopify_filter'); + }); +}); + +describe('UnknownFilter.suggest_nearest', () => { + test('suggests exact match from filters index', () => { + const facts = { graph, tagsIndex: { isTag: () => false }, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'downcase' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.suggest_nearest'); + expect(result.hint_md).toContain('downcase'); + }); + + test('suggests closest match for typos', () => { + const facts = { graph, tagsIndex: { isTag: () => false }, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'doncase' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.suggest_nearest'); + expect(result.hint_md).toContain('downcase'); + }); +}); + +describe('UnknownFilter.generic', () => { + test('fallback for completely unknown filter', () => { + const noMatchIndex = { loaded: true, lookup: () => null, closestMatch: () => null }; + const facts = { graph, tagsIndex: { isTag: () => false }, filtersIndex: noMatchIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'zzz_nonexistent' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.generic'); + }); +}); + +describe('UnknownFilter — edge cases', () => { + test('falls through to .default when filter param is missing', () => { + const facts = { graph, tagsIndex: mockTagsIndex, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: {} }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownFilter.default'); + }); +}); + +describe('UnknownFilter.default catch-all', () => { + test('does NOT preempt .tag_confusion', () => { + const facts = { graph, tagsIndex: mockTagsIndex, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'render' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.tag_confusion'); + }); + + test('does NOT preempt .shopify_filter', () => { + const facts = { graph, tagsIndex: mockTagsIndex, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'money' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.shopify_filter'); + }); + + test('does NOT preempt .suggest_nearest', () => { + const facts = { graph, tagsIndex: { isTag: () => false }, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'downcase' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.suggest_nearest'); + }); + + test('does NOT preempt .generic when filter name was extracted', () => { + const noMatchIndex = { loaded: true, lookup: () => null, closestMatch: () => null }; + const facts = { graph, tagsIndex: { isTag: () => false }, filtersIndex: noMatchIndex }; + const diag = { check: 'UnknownFilter', params: { filter: 'zzz_nonexistent' } }; + const result = runRules(diag, facts); + expect(result.rule_id).toBe('UnknownFilter.generic'); + }); + + test('fires when extraction failed entirely', () => { + const facts = { graph, tagsIndex: mockTagsIndex, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter' }; + const result = runRules(diag, facts); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownFilter.default'); + }); + + test('hint covers typo + Shopify-only escape hatches', () => { + const facts = { graph, tagsIndex: mockTagsIndex, filtersIndex: mockFiltersIndex }; + const diag = { check: 'UnknownFilter', params: {} }; + const result = runRules(diag, facts); + expect(result.hint_md).toContain('lookup'); + expect(result.hint_md).toContain('Shopify'); + }); +}); diff --git a/tests/unit/rules/ValidFrontmatter.test.js b/tests/unit/rules/ValidFrontmatter.test.js new file mode 100644 index 0000000..969ba81 --- /dev/null +++ b/tests/unit/rules/ValidFrontmatter.test.js @@ -0,0 +1,141 @@ +/** + * ValidFrontmatter rule attribution + hint routing per category. + * + * Each category in the rule module dispatches on `diag.params.category` + * (set by the EXTRACTOR in core/diagnostic-record.js). These tests pin: + * - The right rule fires for each category. + * - Hint content surfaces the field/file_type/value the agent needs. + * - The fallback rule covers the unknown shape so every emit is attributed. + */ + +import { describe, test, expect, beforeEach, afterEach } from 'bun:test'; +import { clearRules, registerRules, runRules } from '../../../src/core/rules/engine.js'; +import { rules } from '../../../src/core/rules/ValidFrontmatter.js'; + +const facts = {}; // Frontmatter rules don't depend on fact graph. + +beforeEach(() => { clearRules(); registerRules(rules); }); +afterEach(() => { clearRules(); }); + +function diag(category, extra = {}) { + return { + check: 'ValidFrontmatter', + params: { category, ...extra }, + message: '', + file: 'app/views/pages/x.liquid', + line: 3, + column: 0, + }; +} + +describe('ValidFrontmatter.home_deprecated', () => { + test('attributes home-rename diagnostics', () => { + const r = runRules(diag('home_deprecated'), facts); + expect(r.rule_id).toBe('ValidFrontmatter.home_deprecated'); + expect(r.hint_md).toMatch(/index\.html\.liquid/); + expect(r.confidence).toBe(0.85); + }); +}); + +describe('ValidFrontmatter.missing_required', () => { + test('names the required field and file type in the hint', () => { + const r = runRules(diag('missing_required', { field: 'name', file_type: 'Form' }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.missing_required'); + expect(r.hint_md).toMatch(/`name`/); + expect(r.hint_md).toMatch(/Form/); + expect(r.see_also?.tool).toBe('domain_guide'); + }); + + test('falls back gracefully when file_type is unknown', () => { + const r = runRules(diag('missing_required', { field: 'method' }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.missing_required'); + expect(r.hint_md).toMatch(/`method`/); + }); +}); + +describe('ValidFrontmatter.unknown_field', () => { + test('names the offending key', () => { + const r = runRules(diag('unknown_field', { field: 'cache', file_type: 'Page' }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.unknown_field'); + expect(r.hint_md).toMatch(/`cache`/); + }); +}); + +describe('ValidFrontmatter.deprecated_field', () => { + test('names the deprecated key', () => { + const r = runRules(diag('deprecated_field', { field: 'layout_name' }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.deprecated_field'); + expect(r.hint_md).toMatch(/`layout_name`/); + }); +}); + +describe('ValidFrontmatter.invalid_enum', () => { + test('uppercase HTTP method gets case-canonicalisation guidance', () => { + const r = runRules(diag('invalid_enum', { + field: 'method', + value: 'POST', + allowed: 'get, post, put, delete, patch', + }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.invalid_enum'); + // Canonical-case suggestion ('post') surfaces in the hint. + expect(r.hint_md).toMatch(/`post`/); + }); + + test('truly out-of-range value gets allowed-list guidance', () => { + const r = runRules(diag('invalid_enum', { + field: 'method', + value: 'connect', + allowed: 'get, post, put, delete, patch', + }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.invalid_enum'); + expect(r.hint_md).toMatch(/get, post, put, delete, patch/); + }); +}); + +describe('ValidFrontmatter.layout_false', () => { + test('explains the YAML-boolean footgun', () => { + const r = runRules(diag('layout_false'), facts); + expect(r.rule_id).toBe('ValidFrontmatter.layout_false'); + expect(r.hint_md).toMatch(/`layout: ''`/); + expect(r.confidence).toBe(0.9); + }); +}); + +describe('ValidFrontmatter.layout_missing', () => { + test('app-level layout produces the canonical expected path', () => { + const r = runRules(diag('layout_missing', { layout: 'application' }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.layout_missing'); + expect(r.hint_md).toMatch(/app\/views\/layouts\/application\./); + }); + + test('module-prefixed layout produces the module expected path', () => { + const r = runRules(diag('layout_missing', { layout: 'modules/core/admin' }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.layout_missing'); + expect(r.hint_md).toMatch(/modules\/core\/public\/views\/layouts\/admin\./); + }); +}); + +describe('ValidFrontmatter.association_missing', () => { + test('preserves the upstream label', () => { + const r = runRules(diag('association_missing', { + label: 'Authorization policy', + name: 'guest_only', + }), facts); + expect(r.rule_id).toBe('ValidFrontmatter.association_missing'); + expect(r.hint_md).toMatch(/Authorization policy/); + expect(r.hint_md).toMatch(/`guest_only`/); + }); +}); + +describe('ValidFrontmatter.fallback', () => { + test('attributes unknown shapes (so analytics never see .unmatched)', () => { + const r = runRules(diag('unknown'), facts); + expect(r.rule_id).toBe('ValidFrontmatter.fallback'); + expect(r.confidence).toBe(0.5); + }); + + test('also catches diagnostics with no params at all', () => { + const r = runRules({ check: 'ValidFrontmatter', message: 'mystery message' }, facts); + expect(r.rule_id).toBe('ValidFrontmatter.fallback'); + }); +}); diff --git a/tests/unit/rules/engine.test.js b/tests/unit/rules/engine.test.js new file mode 100644 index 0000000..ee4da50 --- /dev/null +++ b/tests/unit/rules/engine.test.js @@ -0,0 +1,98 @@ +import { describe, test, expect, beforeEach } from 'bun:test'; +import { + registerRule, registerRules, runRules, hasRules, + getRulesForCheck, getAllChecksWithRules, clearRules, ruleCount, +} from '../../../src/core/rules/engine.js'; + +beforeEach(() => clearRules()); + +const makeRule = (overrides = {}) => { + const id = overrides.id ?? 'Test.rule1'; + return { + id, + check: overrides.check ?? 'TestCheck', + priority: overrides.priority ?? 100, + when: overrides.when ?? (() => true), + apply: overrides.apply ?? (() => ({ rule_id: id, hint_md: 'test hint', fixes: [], confidence: 1 })), + }; +}; + +describe('registerRule', () => { + test('registers a valid rule', () => { + registerRule(makeRule()); + expect(hasRules('TestCheck')).toBe(true); + expect(ruleCount()).toBe(1); + }); + + test('throws on missing fields', () => { + expect(() => registerRule({ id: 'x' })).toThrow(); + expect(() => registerRule({ id: 'x', check: 'C' })).toThrow(); + }); + + test('sorts rules by priority', () => { + registerRule(makeRule({ id: 'low', priority: 50 })); + registerRule(makeRule({ id: 'high', priority: 10 })); + registerRule(makeRule({ id: 'mid', priority: 30 })); + const rules = getRulesForCheck('TestCheck'); + expect(rules.map(r => r.id)).toEqual(['high', 'mid', 'low']); + }); +}); + +describe('runRules — first match', () => { + test('returns first matching rule result', () => { + registerRule(makeRule({ id: 'first', priority: 1, apply: () => ({ rule_id: 'first', hint_md: 'first', fixes: [], confidence: 1 }) })); + registerRule(makeRule({ id: 'second', priority: 2, apply: () => ({ rule_id: 'second', hint_md: 'second', fixes: [], confidence: 1 }) })); + const result = runRules({ check: 'TestCheck' }, {}); + expect(result.rule_id).toBe('first'); + }); + + test('skips rules where when() returns false', () => { + registerRule(makeRule({ id: 'skip', priority: 1, when: () => false })); + registerRule(makeRule({ id: 'match', priority: 2 })); + const result = runRules({ check: 'TestCheck' }, {}); + expect(result.rule_id).toBe('match'); + }); + + test('returns null when no rules match', () => { + registerRule(makeRule({ when: () => false })); + expect(runRules({ check: 'TestCheck' }, {})).toBeNull(); + }); + + test('returns null for unknown check', () => { + expect(runRules({ check: 'UnknownCheck' }, {})).toBeNull(); + }); + + test('swallows rule exceptions', () => { + registerRule(makeRule({ when: () => { throw new Error('boom'); } })); + registerRule(makeRule({ id: 'fallback', priority: 200 })); + const result = runRules({ check: 'TestCheck' }, {}); + expect(result.rule_id).toBe('fallback'); + }); +}); + +describe('runRules — multi match', () => { + test('returns all matching results', () => { + registerRule(makeRule({ id: 'a', priority: 1, apply: () => ({ rule_id: 'a', hint_md: 'a', fixes: [], confidence: 1 }) })); + registerRule(makeRule({ id: 'b', priority: 2, apply: () => ({ rule_id: 'b', hint_md: 'b', fixes: [], confidence: 1 }) })); + const results = runRules({ check: 'TestCheck' }, {}, { multiMatch: true }); + expect(results).toHaveLength(2); + expect(results.map(r => r.rule_id)).toEqual(['a', 'b']); + }); +}); + +describe('clearRules', () => { + test('removes all registered rules', () => { + registerRule(makeRule()); + clearRules(); + expect(ruleCount()).toBe(0); + expect(hasRules('TestCheck')).toBe(false); + }); +}); + +describe('getAllChecksWithRules', () => { + test('lists checks that have rules', () => { + registerRule(makeRule({ check: 'A' })); + registerRule(makeRule({ check: 'B', id: 'B.rule' })); + expect(getAllChecksWithRules().sort()).toEqual(['A', 'B']); + }); +}); diff --git a/tests/unit/rules/module-paths.test.js b/tests/unit/rules/module-paths.test.js new file mode 100644 index 0000000..4ad42e8 --- /dev/null +++ b/tests/unit/rules/module-paths.test.js @@ -0,0 +1,117 @@ +import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { + installedModules, + moduleInstalled, + moduleCallPathsByCategory, + moduleCallPaths, +} from '../../../src/core/rules/module-paths.js'; + +let projectDir; + +beforeAll(() => { + projectDir = mkdtempSync(join(tmpdir(), 'modpaths-')); + + const writeFile = (rel, content = '') => { + const abs = join(projectDir, rel); + mkdirSync(join(abs, '..'), { recursive: true }); + writeFileSync(abs, content); + }; + + // core: lib-style layout + writeFile('modules/core/public/lib/commands/execute.liquid'); + writeFile('modules/core/public/lib/commands/email/send/build.liquid'); + writeFile('modules/core/public/lib/commands/email/send/check.liquid'); + writeFile('modules/core/public/lib/queries/users/find.liquid'); + writeFile('modules/core/public/lib/helpers/auth_token.liquid'); + writeFile('modules/core/public/lib/validations/presence.liquid'); + writeFile('modules/core/public/views/partials/widget.liquid'); + + // legacy: views/partials/lib layout + writeFile('modules/legacy/public/views/partials/lib/commands/old_create.liquid'); + writeFile('modules/legacy/public/views/partials/lib/queries/old_find.liquid'); + writeFile('modules/legacy/public/views/partials/banner.liquid'); + + // empty module + mkdirSync(join(projectDir, 'modules', 'empty'), { recursive: true }); +}); + +afterAll(() => { + rmSync(projectDir, { recursive: true, force: true }); +}); + +describe('module-paths.installedModules', () => { + test('lists every directory under modules/', () => { + expect(installedModules(projectDir)).toEqual(['core', 'empty', 'legacy']); + }); + + test('returns [] when modules/ is missing', () => { + expect(installedModules('/nonexistent')).toEqual([]); + }); + + test('returns [] when projectDir is null', () => { + expect(installedModules(null)).toEqual([]); + }); +}); + +describe('module-paths.moduleInstalled', () => { + test('true for present module', () => { + expect(moduleInstalled(projectDir, 'core')).toBe(true); + }); + + test('false for absent module', () => { + expect(moduleInstalled(projectDir, 'ghost')).toBe(false); + }); + + test('false on null inputs', () => { + expect(moduleInstalled(null, 'core')).toBe(false); + expect(moduleInstalled(projectDir, null)).toBe(false); + }); +}); + +describe('module-paths.moduleCallPathsByCategory', () => { + test('groups core lib exports by category with full call_paths', () => { + const out = moduleCallPathsByCategory(projectDir, 'core'); + expect(out.commands).toEqual([ + 'modules/core/commands/email/send/build', + 'modules/core/commands/email/send/check', + 'modules/core/commands/execute', + ]); + expect(out.queries).toEqual(['modules/core/queries/users/find']); + expect(out.helpers).toEqual(['modules/core/helpers/auth_token']); + expect(out.validations).toEqual(['modules/core/validations/presence']); + expect(out.partials).toContain('modules/core/widget'); + }); + + test('falls back to views/partials/lib for legacy layout', () => { + const out = moduleCallPathsByCategory(projectDir, 'legacy'); + expect(out.commands).toEqual(['modules/legacy/commands/old_create']); + expect(out.queries).toEqual(['modules/legacy/queries/old_find']); + expect(out.partials).toContain('modules/legacy/banner'); + }); + + test('returns empty buckets for an empty module', () => { + const out = moduleCallPathsByCategory(projectDir, 'empty'); + expect(out.commands).toEqual([]); + expect(out.queries).toEqual([]); + expect(out.partials).toEqual([]); + }); + + test('returns empty buckets when module is absent', () => { + const out = moduleCallPathsByCategory(projectDir, 'ghost'); + expect(Object.values(out).every(v => v.length === 0)).toBe(true); + }); +}); + +describe('module-paths.moduleCallPaths', () => { + test('flattens every callable across categories', () => { + const flat = moduleCallPaths(projectDir, 'core'); + expect(flat).toContain('modules/core/commands/execute'); + expect(flat).toContain('modules/core/queries/users/find'); + expect(flat).toContain('modules/core/helpers/auth_token'); + expect(flat).toContain('modules/core/validations/presence'); + expect(flat).toContain('modules/core/widget'); + }); +}); diff --git a/tests/unit/rules/queries.test.js b/tests/unit/rules/queries.test.js new file mode 100644 index 0000000..90a54a4 --- /dev/null +++ b/tests/unit/rules/queries.test.js @@ -0,0 +1,206 @@ +import { describe, test, expect } from 'bun:test'; +import { + nearestByLevenshtein, partialNames, commandPaths, queryPaths, + partialsReachableFrom, dependentsOf, translationKeysForLocale, + schemaNames, fileExists, classifyPath, stripLocalePrefix, +} from '../../../src/core/rules/queries.js'; +import { buildFactGraph } from '../../../src/core/project-fact-graph.js'; + +const FIXTURE_MAP = { + pages: { + 'blog_posts:get': { path: 'app/views/pages/blog_posts/index.html.liquid', slug: 'blog_posts', method: 'get', renders: ['blog_posts/list'], function_calls: [] }, + }, + partials: { + 'blog_posts/list': { path: 'app/views/partials/blog_posts/list.liquid', params: [], renders: ['blog_posts/card'], function_calls: [], rendered_by: [] }, + 'blog_posts/card': { path: 'app/views/partials/blog_posts/card.liquid', params: [], renders: [], function_calls: [], rendered_by: [] }, + 'blog_posts/form': { path: 'app/views/partials/blog_posts/form.liquid', params: [], renders: [], function_calls: [], rendered_by: [] }, + }, + commands: { + 'app/lib/commands/blog_posts/create.liquid': { params: [], phases: ['main'], graphql_calls: [{ queryName: 'blog_posts/create' }], function_calls: [] }, + }, + queries: { + 'app/lib/queries/blog_posts/find.liquid': { params: [], graphql_calls: [{ queryName: 'blog_posts/find' }], function_calls: [] }, + }, + graphql: { + 'blog_posts/create': { operation: 'mutation', name: 'CreateBlogPost', args: [], table: 'blog_post' }, + 'blog_posts/find': { operation: 'query', name: 'FindBlogPost', args: [], table: 'blog_post' }, + }, + schema: { + 'blog_post': { path: 'app/schema/blog_post.yml', properties: [{ name: 'title', type: 'string' }] }, + }, + layouts: {}, + translations: { en: { 'app.title': 'Blog' } }, + assets: ['styles/app.css'], +}; + +const graph = buildFactGraph(FIXTURE_MAP); + +describe('nearestByLevenshtein', () => { + test('finds closest matches', () => { + const result = nearestByLevenshtein('blog_posts/lst', ['blog_posts/list', 'blog_posts/card', 'blog_posts/form']); + expect(result[0].name).toBe('blog_posts/list'); + expect(result[0].distance).toBe(1); + }); + + test('returns empty for no candidates', () => { + expect(nearestByLevenshtein('test', [])).toEqual([]); + }); + + test('filters by max distance', () => { + const result = nearestByLevenshtein('x', ['abcdefghij']); + expect(result).toEqual([]); + }); + + test('returns up to k results', () => { + const candidates = ['aa', 'ab', 'ac', 'ad', 'ae', 'af']; + const result = nearestByLevenshtein('aa', candidates, 3); + expect(result.length).toBeLessThanOrEqual(3); + }); +}); + +describe('node queries', () => { + test('partialNames returns all partial keys', () => { + const names = partialNames(graph); + expect(names).toContain('blog_posts/list'); + expect(names).toContain('blog_posts/card'); + expect(names).toContain('blog_posts/form'); + }); + + test('commandPaths returns command keys', () => { + const paths = commandPaths(graph); + expect(paths).toContain('app/lib/commands/blog_posts/create.liquid'); + }); + + test('queryPaths returns query keys', () => { + const paths = queryPaths(graph); + expect(paths).toContain('app/lib/queries/blog_posts/find.liquid'); + }); + + test('schemaNames returns schema keys', () => { + expect(schemaNames(graph)).toContain('blog_post'); + }); + + test('translationKeysForLocale returns keys', () => { + expect(translationKeysForLocale(graph, 'en')).toContain('app.title'); + }); + + test('translationKeysForLocale strips the leading `.` prefix', () => { + // Realistic shape: `flattenYaml` over a properly-rooted en.yml emits + // keys prefixed with `en.` because the YAML root is the locale name. + const realistic = buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, + translations: { en: { 'en.app.user.title': 'X', 'en.app.user.name': 'Y' } }, + assets: [], + }); + const keys = translationKeysForLocale(realistic, 'en'); + expect(keys).toContain('app.user.title'); + expect(keys).toContain('app.user.name'); + expect(keys.every(k => !k.startsWith('en.'))).toBe(true); + }); + + test('translationKeysForLocale leaves bare keys untouched', () => { + // Mis-shaped YAML (no locale wrapper) flattens to bare `app.title`. + // The helper should NOT invent a prefix to strip. + const bare = buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, graphql: {}, schema: {}, layouts: {}, + translations: { en: { 'app.title': 'X' } }, + assets: [], + }); + expect(translationKeysForLocale(bare, 'en')).toEqual(['app.title']); + }); +}); + +describe('stripLocalePrefix', () => { + test('strips matching `.` prefix', () => { + expect(stripLocalePrefix('en.app.foo', 'en')).toBe('app.foo'); + }); + + test('leaves a bare key unchanged', () => { + expect(stripLocalePrefix('app.foo', 'en')).toBe('app.foo'); + }); + + test('does not strip a different locale', () => { + expect(stripLocalePrefix('pl.app.foo', 'en')).toBe('pl.app.foo'); + }); + + test('handles edge inputs without throwing', () => { + expect(stripLocalePrefix('', 'en')).toBe(''); + expect(stripLocalePrefix(null, 'en')).toBe(null); + expect(stripLocalePrefix(undefined, 'en')).toBe(undefined); + }); + + test('default locale is `en`', () => { + expect(stripLocalePrefix('en.app.foo')).toBe('app.foo'); + }); +}); + +describe('partialsReachableFrom', () => { + test('follows render edges transitively', () => { + const reachable = partialsReachableFrom(graph, 'app/views/pages/blog_posts/index.html.liquid'); + expect(reachable).toContain('blog_posts/list'); + expect(reachable).toContain('blog_posts/card'); + }); + + test('returns empty for leaf node', () => { + expect(partialsReachableFrom(graph, 'app/views/partials/blog_posts/card.liquid')).toEqual([]); + }); +}); + +describe('dependentsOf', () => { + test('returns callers of a partial', () => { + const deps = dependentsOf(graph, 'app/views/partials/blog_posts/list.liquid'); + expect(deps).toContain('app/views/pages/blog_posts/index.html.liquid'); + }); +}); + +describe('fileExists', () => { + test('returns true for known path', () => { + expect(fileExists(graph, 'app/views/partials/blog_posts/card.liquid')).toBe(true); + }); + + test('returns false for unknown path', () => { + expect(fileExists(graph, 'app/views/partials/nope.liquid')).toBe(false); + }); +}); + +describe('classifyPath', () => { + test('classifies partial', () => { + expect(classifyPath('blog_posts/card')).toEqual({ type: 'partial', path: 'app/views/partials/blog_posts/card.liquid' }); + }); + + test('classifies command', () => { + expect(classifyPath('commands/blog_posts/create')).toEqual({ type: 'command', path: 'app/lib/commands/blog_posts/create.liquid' }); + }); + + test('flags `lib/commands/` as an invalid prefix and exposes the corrected name', () => { + // Function-tag paths resolve under the partial search paths + // (`app/views/partials/`, `app/lib/`), so a literal `lib/` prefix + // would expand to `app/lib/lib/...` which never exists. Treating the + // prefix as "optional" (the prior behaviour) hid the bug from agents. + expect(classifyPath('lib/commands/blog_posts/create')).toEqual({ + type: 'invalid_lib_prefix', + path: null, + correctedName: 'commands/blog_posts/create', + }); + }); + + test('classifies query', () => { + expect(classifyPath('queries/blog_posts/find')).toEqual({ type: 'query', path: 'app/lib/queries/blog_posts/find.liquid' }); + }); + + test('flags `lib/queries/` as an invalid prefix and exposes the corrected name', () => { + expect(classifyPath('lib/queries/blog_posts/find')).toEqual({ + type: 'invalid_lib_prefix', + path: null, + correctedName: 'queries/blog_posts/find', + }); + }); + + test('classifies module', () => { + expect(classifyPath('modules/user/helpers/auth')).toEqual({ type: 'module', path: null }); + }); + + test('handles null', () => { + expect(classifyPath(null)).toEqual({ type: 'unknown', path: null }); + }); +}); diff --git a/tests/unit/session-events.test.js b/tests/unit/session-events.test.js new file mode 100644 index 0000000..f687275 --- /dev/null +++ b/tests/unit/session-events.test.js @@ -0,0 +1,347 @@ +/** + * session-events unit tests — pins the event envelope, per-kind payload + * validation, version upgrader contract, and append-only NDJSON I/O. + * + * The on-disk NDJSON log is the canonical history; the reducer's projection + * is rebuildable from it. These tests make sure write/read are symmetric so + * Phase A's acceptance gate (events.ndjson replay reproduces getStatus) + * has a sound foundation. + */ + +import { describe, it, expect } from 'bun:test'; +import { mkdtempSync, rmSync, readFileSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { + EVENT_VERSION, + KNOWN_KINDS, + KIND_SCHEMAS, + makeEvent, + validateEvent, + readEvent, + readEventLog, + createEventWriter, +} from '../../src/core/session-events.js'; + +const FIXED_TS = '2025-01-01T00:00:00.000Z'; +const SID = 'test-session-1'; + +function workDir() { + const dir = mkdtempSync(join(tmpdir(), 'pos-events-')); + return { dir, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; +} + +describe('session-events: schema registry', () => { + it('exposes the expected event kinds', () => { + expect(KNOWN_KINDS).toEqual([ + 'server_start', + 'server_stop', + 'pos_cli_resolved', + 'lsp_event', + 'index_event', + 'fs_change', + 'tool_call', + 'validator_emit', + 'log', + 'cac_decision', + ]); + for (const kind of KNOWN_KINDS) { + expect(KIND_SCHEMAS[kind]).toBeDefined(); + } + }); + + it('current version is at least 1', () => { + expect(EVENT_VERSION).toBeGreaterThanOrEqual(1); + }); +}); + +describe('session-events: makeEvent + validateEvent', () => { + it('builds a server_start event with envelope + payload', () => { + const e = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'server_start', + payload: { project_dir: '/x', version: '0.0.0', http_port: 13800, started_at: FIXED_TS }, + }); + expect(e.v).toBe(EVENT_VERSION); + expect(e.session_id).toBe(SID); + expect(e.ts).toBe(FIXED_TS); + expect(e.kind).toBe('server_start'); + expect(e.project_dir).toBe('/x'); + expect(e.http_port).toBe(13800); + }); + + it('rejects an unknown kind', () => { + expect(() => makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'bogus_kind', payload: {}, + })).toThrow(/unknown kind/i); + }); + + it('rejects missing session_id / ts', () => { + expect(() => makeEvent({ + session_id: '', ts: FIXED_TS, kind: 'log', payload: { message: 'x' }, + })).toThrow(); + expect(() => makeEvent({ + session_id: SID, ts: '', kind: 'log', payload: { message: 'x' }, + })).toThrow(); + }); + + it('rejects a payload that fails the per-kind schema', () => { + expect(() => makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'lsp_event', + payload: { phase: 'not-a-phase' }, + })).toThrow(); + }); + + it('validateEvent returns a frozen object', () => { + const e = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'log', payload: { level: 'info', message: 'hi' }, + }); + expect(Object.isFrozen(e)).toBe(true); + }); + + it('tool_call payload is permissive on input/output (validated at tool boundary)', () => { + const e = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'tool_call', + payload: { + tool: 'validate_code', + duration_ms: 12, + success: true, + input: { file_path: 'a.liquid', anything: { nested: true } }, + output: { errors: [], warnings: [{ check: 'X' }] }, + }, + }); + expect(e.input.anything.nested).toBe(true); + expect(e.output.warnings[0].check).toBe('X'); + }); + + // Regression: prior to registering the schema, every cac_decision emit + // threw "unknown kind" inside makeEvent, the throw was swallowed by the + // caller's try/catch, and the predictor's audit trail was silently lost + // on every restart. Pin the happy path AND every rejection edge so a + // future refactor can't reopen the hole quietly. + it('builds a cac_decision event with envelope + payload', () => { + const e = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'cac_decision', + payload: { + file: 'app/views/pages/index.liquid', + rule_id: 'MissingPartial.create_file', + check: 'MissingPartial', + severity: 'error', + file_domain: 'pages', + p_adopted: 0.18, + p_lower: 0.05, + p_upper: 0.45, + n_samples: 7, + feature: 'rule_id', + decision: 'downgrade', + reason: 'below_threshold', + mode: 'shadow', + }, + }); + expect(e.kind).toBe('cac_decision'); + expect(e.rule_id).toBe('MissingPartial.create_file'); + expect(e.feature).toBe('rule_id'); + expect(e.decision).toBe('downgrade'); + expect(e.mode).toBe('shadow'); + }); + + it('cac_decision accepts the no-signal `prior` shape with null probabilities', () => { + const e = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'cac_decision', + payload: { + file: 'app/views/pages/x.liquid', + rule_id: 'NewCheck.unmatched', + check: 'NewCheck', + severity: 'warning', + file_domain: null, + p_adopted: 0.5, + p_lower: null, + p_upper: null, + n_samples: 0, + feature: 'prior', + decision: 'allow', + reason: 'no_signal', + mode: 'shadow', + }, + }); + expect(e.feature).toBe('prior'); + expect(e.p_lower).toBeNull(); + }); + + it('cac_decision rejects a payload that smuggles the envelope `ts` key', () => { + // This is the exact collision that used to drop events at runtime — the + // ring entry carried `ts` (its own timestamp) and was passed verbatim + // as a payload, hitting `ENVELOPE_KEYS` in makeEvent. + expect(() => makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'cac_decision', + payload: { + ts: FIXED_TS, + rule_id: 'X', severity: 'error', + p_adopted: 0.5, p_lower: null, p_upper: null, + n_samples: 0, feature: 'prior', decision: 'allow', reason: '', + mode: 'shadow', + }, + })).toThrow(/reserved envelope key/i); + }); + + it('cac_decision rejects an unknown decision value', () => { + expect(() => makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'cac_decision', + payload: { + rule_id: 'X', severity: 'error', + p_adopted: 0.5, p_lower: null, p_upper: null, + n_samples: 0, feature: 'prior', decision: 'mute_forever', reason: '', + mode: 'shadow', + }, + })).toThrow(); + }); + + it('cac_decision round-trips through readEvent / writeEvent', () => { + const written = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'cac_decision', + payload: { + file: 'a.liquid', + rule_id: 'PartialCallArguments.required_render', + check: 'PartialCallArguments', + severity: 'warning', + file_domain: 'partials', + p_adopted: 0.17, p_lower: 0.03, p_upper: 0.41, + n_samples: 8, feature: 'rule_id+domain', + decision: 'downgrade', reason: 'below_threshold', + mode: 'active', + }, + }); + const back = readEvent(JSON.stringify(written)); + expect(back).toEqual(written); + }); +}); + +describe('session-events: readEvent', () => { + it('round-trips JSON encoding', () => { + const written = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'index_event', + payload: { index: 'objects', status: 'ready', count: 42 }, + }); + const back = readEvent(JSON.stringify(written)); + expect(back).toEqual(written); + }); + + it('rejects missing version', () => { + expect(() => readEvent(JSON.stringify({ kind: 'log', session_id: SID, ts: FIXED_TS, message: 'x' }))) + .toThrow(/missing version/i); + }); + + it('rejects a future event version', () => { + const fake = JSON.stringify({ v: EVENT_VERSION + 1, session_id: SID, ts: FIXED_TS, kind: 'log', message: 'x' }); + expect(() => readEvent(fake)).toThrow(/newer than reader/i); + }); + + it('rejects malformed JSON', () => { + expect(() => readEvent('{not-json')).toThrow(/invalid JSON/i); + }); + + it('rejects empty input', () => { + expect(() => readEvent('')).toThrow(/empty/i); + }); +}); + +describe('session-events: createEventWriter + readEventLog', () => { + it('appends events and reads them back in order', () => { + const { dir, cleanup } = workDir(); + try { + const file = join(dir, 'sub', 'events.ndjson'); + const writer = createEventWriter(file); + + const e1 = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'server_start', + payload: { project_dir: '/p', version: '1.0.0', started_at: FIXED_TS }, + }); + const e2 = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'tool_call', + payload: { tool: 'lookup', duration_ms: 5, success: true }, + }); + writer.append(e1); + writer.append(e2); + writer.close(); + + const events = readEventLog(file); + expect(events).toHaveLength(2); + expect(events[0].kind).toBe('server_start'); + expect(events[1].tool).toBe('lookup'); + } finally { + cleanup(); + } + }); + + it('returns [] for a missing file', () => { + const { dir, cleanup } = workDir(); + try { + expect(readEventLog(join(dir, 'nope.ndjson'))).toEqual([]); + } finally { + cleanup(); + } + }); + + it('skips malformed lines via onError callback', () => { + const { dir, cleanup } = workDir(); + try { + const file = join(dir, 'mixed.ndjson'); + const good = makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'log', + payload: { level: 'info', message: 'ok' }, + }); + writeFileSync(file, [ + JSON.stringify(good), + 'broken-line', + JSON.stringify({ v: EVENT_VERSION, session_id: SID, ts: FIXED_TS, kind: 'unknown_kind' }), + JSON.stringify(good), + ].join('\n') + '\n'); + + const errors = []; + const events = readEventLog(file, { onError: (e) => errors.push(e) }); + expect(events).toHaveLength(2); + expect(errors.length).toBe(2); + expect(errors[0].line).toBe(2); + } finally { + cleanup(); + } + }); + + it('writer cannot append after close', () => { + const { dir, cleanup } = workDir(); + try { + const writer = createEventWriter(join(dir, 'e.ndjson')); + writer.close(); + expect(writer.isClosed).toBe(true); + expect(() => writer.append(makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'log', payload: { message: 'x' }, + }))).toThrow(/closed/i); + } finally { + cleanup(); + } + }); + + it('persists raw line content as valid NDJSON (one event per line)', () => { + const { dir, cleanup } = workDir(); + try { + const file = join(dir, 'raw.ndjson'); + const writer = createEventWriter(file); + writer.append(makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'log', + payload: { level: 'info', message: 'one' }, + })); + writer.append(makeEvent({ + session_id: SID, ts: FIXED_TS, kind: 'log', + payload: { level: 'warn', message: 'two' }, + })); + writer.close(); + + const lines = readFileSync(file, 'utf-8').trim().split('\n'); + expect(lines).toHaveLength(2); + for (const line of lines) { + expect(() => JSON.parse(line)).not.toThrow(); + } + } finally { + cleanup(); + } + }); +}); diff --git a/tests/unit/session-state.test.js b/tests/unit/session-state.test.js new file mode 100644 index 0000000..243ce1c --- /dev/null +++ b/tests/unit/session-state.test.js @@ -0,0 +1,452 @@ +/** + * session-state unit tests — pins the pure reducer that backs every dashboard + * value and every analytics derivation downstream. Two separate guarantees: + * + * 1. Each event kind reduces the state correctly (per-handler tests). + * 2. Replay equivalence: applying the same event sequence twice — once + * incrementally, once via `replay()` — yields byte-identical state. + * This is the contract the Phase A acceptance gate rides on. + * + * The reducer is forbidden from reading the clock or making random calls, + * so deepEqual on the full state is a meaningful invariant. + */ + +import { describe, it, expect } from 'bun:test'; +import { initialState, applyEvent, replay } from '../../src/core/session-state.js'; +import { makeEvent } from '../../src/core/session-events.js'; + +const SID = 'reducer-test'; +const T = (n) => `2025-01-01T00:00:0${n}.000Z`; + +function ev(kind, payload, ts = T(0)) { + return makeEvent({ session_id: SID, ts, kind, payload }); +} + +describe('session-state: initialState', () => { + it('returns a fresh object with all expected keys', () => { + const s = initialState(); + expect(Object.keys(s).sort()).toEqual([ + '_event_count', + 'by_tool', + 'check_effectiveness', + 'check_frequency', + 'enrich_history', + 'file_history', + 'hint_effectiveness', + 'indexes', + 'last_analysis', + 'lsp', + 'pending', + 'pipeline_traces', + 'pos_cli', + 'scaffold_runs', + 'server', + 'validated_plan', + 'validator_emissions', + ]); + expect(s._event_count).toBe(0); + expect(s.pending.files).toEqual([]); + expect(s.indexes.schema.status).toBe('pending'); + }); + + it('returns a NEW object each call (no shared references)', () => { + const a = initialState(); + const b = initialState(); + expect(a).not.toBe(b); + expect(a.pending).not.toBe(b.pending); + a.pending.files.push('x'); + expect(b.pending.files).toEqual([]); + }); +}); + +describe('session-state: lifecycle handlers', () => { + it('server_start populates server.* and clears stop state', () => { + let s = initialState(); + s.server.stopped = true; + s = applyEvent(s, ev('server_start', { + project_dir: '/p', version: '1.2.3', http_port: 13800, started_at: T(0), + })); + expect(s.server.started_at).toBe(T(0)); + expect(s.server.version).toBe('1.2.3'); + expect(s.server.project_dir).toBe('/p'); + expect(s.server.http_port).toBe(13800); + expect(s.server.stopped).toBe(false); + expect(s._event_count).toBe(1); + }); + + it('server_stop sets stopped + reason', () => { + const s = applyEvent(initialState(), ev('server_stop', { reason: 'SIGINT' })); + expect(s.server.stopped).toBe(true); + expect(s.server.stop_reason).toBe('SIGINT'); + }); + + it('pos_cli_resolved sets full record', () => { + const s = applyEvent(initialState(), ev('pos_cli_resolved', { + found: true, path: '/usr/bin/pos-cli', data_dir: '/data', + })); + expect(s.pos_cli).toEqual({ found: true, path: '/usr/bin/pos-cli', data_dir: '/data', error: null }); + }); + + it('lsp_event ready/crash/restart cycle', () => { + let s = initialState(); + s = applyEvent(s, ev('lsp_event', { phase: 'ready', duration_ms: 250 })); + expect(s.lsp.ready).toBe(true); + expect(s.lsp.last_ready_ms).toBe(250); + + s = applyEvent(s, ev('lsp_event', { phase: 'warmed_up', duration_ms: 1000, index_ready: 3 })); + expect(s.lsp.last_warmup_ms).toBe(1000); + expect(s.lsp.last_warmup_index_ready).toBe(3); + + s = applyEvent(s, ev('lsp_event', { phase: 'crash', code: 1, signal: 'SIGTERM', restart_count: 2 }, T(2))); + expect(s.lsp.ready).toBe(false); + expect(s.lsp.restart_count).toBe(2); + expect(s.lsp.last_crash).toEqual({ code: 1, signal: 'SIGTERM', ts: T(2) }); + + s = applyEvent(s, ev('lsp_event', { phase: 'restart_failed', error: 'boom' })); + expect(s.lsp.last_error).toBe('boom'); + }); + + it('index_event populates per-index entries', () => { + let s = initialState(); + s = applyEvent(s, ev('index_event', { index: 'schema', status: 'ready', queries: 10, mutations: 4 })); + expect(s.indexes.schema).toEqual({ status: 'ready', queries: 10, mutations: 4 }); + + s = applyEvent(s, ev('index_event', { index: 'objects', status: 'failed', error: 'nope' })); + expect(s.indexes.objects).toEqual({ status: 'failed', error: 'nope' }); + + // 'all' marks every index failed + s = applyEvent(s, ev('index_event', { index: 'all', status: 'failed', error: 'data dir missing' })); + for (const k of ['schema', 'objects', 'filters', 'tags']) { + expect(s.indexes[k].status).toBe('failed'); + } + }); + + it('fs_change and log are no-ops on the projection', () => { + const s0 = initialState(); + let s = applyEvent(s0, ev('fs_change', { path: 'app/views/pages/x.liquid', op: 'update' })); + expect(s._event_count).toBe(1); // counter still increments + s = applyEvent(s, ev('log', { level: 'info', message: 'hi' })); + expect(s._event_count).toBe(2); + // No other state changes + const cmp = { ...s, _event_count: 0 }; + expect(cmp).toEqual({ ...s0, _event_count: 0 }); + }); +}); + +describe('session-state: tool_call / validate_intent', () => { + it('writes pending + validated_plan on success', () => { + const s = applyEvent(initialState(), ev('tool_call', { + tool: 'validate_intent', + duration_ms: 30, + success: true, + input: {}, + output: { + ok: true, + plan_id: 'p1', + validated_at: T(1), + pending_files: ['app/views/pages/blog.liquid', 'app/views/partials/blog/form.liquid'], + pending_translations: ['blog.title'], + write_directly: false, + }, + })); + expect(s.pending.plan_id).toBe('p1'); + expect(s.pending.files).toEqual([ + 'app/views/pages/blog.liquid', + 'app/views/partials/blog/form.liquid', + ]); + expect(s.pending.pages).toEqual(['app/views/pages/blog.liquid']); + expect(s.pending.translations).toEqual(['blog.title']); + expect(s.validated_plan.plan_id).toBe('p1'); + expect(s.validated_plan.source).toBe('manual'); + expect(s.validated_plan.validated_files).toEqual([]); + }); + + it('pre-marks files validated when triggered by scaffold_output', () => { + const s = applyEvent(initialState(), ev('tool_call', { + tool: 'validate_intent', + duration_ms: 30, + success: true, + input: { scaffold_output: { files: [] } }, + output: { + ok: true, + plan_id: 'p2', + pending_files: ['app/views/pages/x.liquid'], + pending_translations: [], + }, + })); + expect(s.validated_plan.source).toBe('scaffold'); + expect(s.validated_plan.validated_files).toEqual(['app/views/pages/x.liquid']); + }); + + it('does NOT mutate state on ok=false', () => { + const s0 = initialState(); + const s = applyEvent(s0, ev('tool_call', { + tool: 'validate_intent', + duration_ms: 30, + success: true, + input: {}, + output: { ok: false, errors: [{ message: 'bad' }] }, + })); + expect(s.pending).toEqual(s0.pending); + expect(s.validated_plan).toBeNull(); + }); +}); + +describe('session-state: tool_call / validate_code', () => { + function vc(filePath, errors = [], warnings = [], status = 'ok') { + return ev('tool_call', { + tool: 'validate_code', + duration_ms: 50, + success: true, + input: { file_path: filePath }, + output: { errors, warnings, status }, + }); + } + + it('initializes file_history on first call', () => { + const s = applyEvent(initialState(), vc('a.liquid', + [{ check: 'MissingPartial' }], + [{ check: 'UnusedAssign' }], + )); + expect(s.file_history['a.liquid']).toEqual({ + calls: 1, + consecutive_non_decreasing: 0, + last_error_count: 1, + last_warning_count: 1, + last_checks: ['MissingPartial', 'UnusedAssign'], + prev_checks: [], + }); + expect(s.check_frequency).toEqual({ MissingPartial: 1, UnusedAssign: 1 }); + expect(s.by_tool.validate_code).toEqual({ calls: 1, errors: 0, total_ms: 50 }); + }); + + it('tracks consecutive_non_decreasing across repeated calls', () => { + let s = initialState(); + s = applyEvent(s, vc('a.liquid', [{ check: 'X' }])); + s = applyEvent(s, vc('a.liquid', [{ check: 'X' }, { check: 'Y' }])); + s = applyEvent(s, vc('a.liquid', [{ check: 'X' }, { check: 'Y' }])); + expect(s.file_history['a.liquid'].calls).toBe(3); + expect(s.file_history['a.liquid'].consecutive_non_decreasing).toBe(2); + }); + + it('updates check_effectiveness across consecutive calls (fixed vs stuck)', () => { + let s = initialState(); + s = applyEvent(s, vc('a.liquid', [{ check: 'A' }, { check: 'B' }])); + s = applyEvent(s, vc('a.liquid', [{ check: 'B' }])); // A fixed, B stuck + expect(s.check_effectiveness).toEqual({ + A: { fixed: 1, stuck: 0 }, + B: { fixed: 0, stuck: 1 }, + }); + }); + + it('marks file as validated in plan when status != error', () => { + let s = initialState(); + s = applyEvent(s, ev('tool_call', { + tool: 'validate_intent', + duration_ms: 5, success: true, input: {}, + output: { ok: true, plan_id: 'p1', pending_files: ['a.liquid'], pending_translations: [] }, + })); + s = applyEvent(s, vc('a.liquid', [], [{ check: 'W' }], 'warning')); + expect(s.validated_plan.validated_files).toEqual(['a.liquid']); + + // Re-validating doesn't double-add + s = applyEvent(s, vc('a.liquid', [], [], 'ok')); + expect(s.validated_plan.validated_files).toEqual(['a.liquid']); + }); + + it('does NOT mark validated when status === error', () => { + let s = initialState(); + s = applyEvent(s, ev('tool_call', { + tool: 'validate_intent', duration_ms: 5, success: true, input: {}, + output: { ok: true, plan_id: 'p1', pending_files: ['a.liquid'], pending_translations: [] }, + })); + s = applyEvent(s, vc('a.liquid', [{ check: 'E' }], [], 'error')); + expect(s.validated_plan.validated_files).toEqual([]); + }); +}); + +describe('session-state: tool_call / enrich_error → validate_code correlation', () => { + it('queues enrich and consumes on validate_code, computing hint effectiveness', () => { + let s = initialState(); + s = applyEvent(s, ev('tool_call', { + tool: 'enrich_error', duration_ms: 2, success: true, + input: { file_path: 'a.liquid', check_name: 'MissingPartial' }, + }, T(0))); + expect(s.enrich_history).toHaveLength(1); + + // validate_code on same file with the error STILL present → hinted but not fixed + s = applyEvent(s, ev('tool_call', { + tool: 'validate_code', duration_ms: 10, success: true, + input: { file_path: 'a.liquid' }, + output: { errors: [{ check: 'MissingPartial' }], warnings: [] }, + }, T(1))); + expect(s.enrich_history).toEqual([]); + expect(s.hint_effectiveness.MissingPartial).toEqual({ hinted: 1, fixed_after_hint: 0 }); + + // enrich again then validate_code where error is gone → fixed_after_hint + s = applyEvent(s, ev('tool_call', { + tool: 'enrich_error', duration_ms: 2, success: true, + input: { file_path: 'a.liquid', check_name: 'MissingPartial' }, + }, T(2))); + s = applyEvent(s, ev('tool_call', { + tool: 'validate_code', duration_ms: 10, success: true, + input: { file_path: 'a.liquid' }, + output: { errors: [], warnings: [] }, + }, T(3))); + expect(s.hint_effectiveness.MissingPartial).toEqual({ hinted: 2, fixed_after_hint: 1 }); + }); + + it('only consumes enrich entries matching the file_path', () => { + let s = initialState(); + s = applyEvent(s, ev('tool_call', { + tool: 'enrich_error', duration_ms: 1, success: true, + input: { file_path: 'b.liquid', check_name: 'X' }, + })); + s = applyEvent(s, ev('tool_call', { + tool: 'validate_code', duration_ms: 1, success: true, + input: { file_path: 'a.liquid' }, + output: { errors: [], warnings: [] }, + })); + expect(s.enrich_history).toHaveLength(1); + }); +}); + +describe('session-state: tool_call / scaffold', () => { + it('logs scaffold_runs and clears pending on write', () => { + let s = initialState(); + // First, set pending via validate_intent so we can observe it being cleared. + s = applyEvent(s, ev('tool_call', { + tool: 'validate_intent', duration_ms: 1, success: true, input: {}, + output: { ok: true, plan_id: 'p1', pending_files: ['app/views/pages/x.liquid'], pending_translations: [] }, + }, T(0))); + expect(s.pending.files.length).toBe(1); + + s = applyEvent(s, ev('tool_call', { + tool: 'scaffold', duration_ms: 100, success: true, + input: { write: true, model: 'note', type: 'crud' }, + output: { files: [{ path: 'a.liquid' }, { path: 'b.liquid' }], written: ['a.liquid', 'b.liquid'] }, + }, T(1))); + expect(s.scaffold_runs).toHaveLength(1); + expect(s.scaffold_runs[0]).toEqual({ + ts: T(1), model: 'note', type: 'crud', + files: ['a.liquid', 'b.liquid'], + written: ['a.liquid', 'b.liquid'], + }); + expect(s.pending.files).toEqual([]); + expect(s.pending.translations).toEqual([]); + expect(s.pending.pages).toEqual([]); + expect(s.pending.plan_id).toBeNull(); + }); + + it('does not clear pending on dry-run (write=false)', () => { + let s = initialState(); + s = applyEvent(s, ev('tool_call', { + tool: 'validate_intent', duration_ms: 1, success: true, input: {}, + output: { ok: true, plan_id: 'p1', pending_files: ['x.liquid'], pending_translations: ['t.k'] }, + })); + s = applyEvent(s, ev('tool_call', { + tool: 'scaffold', duration_ms: 100, success: true, + input: { write: false }, + output: { files: [{ path: 'x.liquid' }], written: [] }, + })); + expect(s.pending.files).toEqual(['x.liquid']); + expect(s.pending.translations).toEqual(['t.k']); + }); +}); + +describe('session-state: tool_call / analyze_project', () => { + it('records last_analysis snapshot', () => { + const s = applyEvent(initialState(), ev('tool_call', { + tool: 'analyze_project', duration_ms: 200, success: true, + input: {}, output: { total_errors: 3, total_warnings: 7 }, + }, T(5))); + expect(s.last_analysis).toEqual({ + ts: T(5), total_errors: 3, total_warnings: 7, diagnostics: null, + }); + }); +}); + +describe('session-state: by_tool counters', () => { + it('counts errors when success=false', () => { + let s = initialState(); + s = applyEvent(s, ev('tool_call', { tool: 'lookup', duration_ms: 1, success: true })); + s = applyEvent(s, ev('tool_call', { tool: 'lookup', duration_ms: 2, success: false, error: 'x' })); + s = applyEvent(s, ev('tool_call', { tool: 'lookup', duration_ms: 3, success: true })); + expect(s.by_tool.lookup).toEqual({ calls: 3, errors: 1, total_ms: 6 }); + }); +}); + +describe('session-state: validator_emit', () => { + it('appends to validator_emissions ring buffer', () => { + let s = initialState(); + for (let i = 0; i < 3; i++) { + s = applyEvent(s, ev('validator_emit', { + fp: `fp-${i}`, file: 'a.liquid', hint_rule_id: 'R', hint_md_hash: 'h', + proposed_fixes: [], + }, T(i))); + } + expect(s.validator_emissions).toHaveLength(3); + expect(s.validator_emissions[2].fp).toBe('fp-2'); + }); +}); + +describe('session-state: replay equivalence', () => { + it('replay and incremental application produce identical state', () => { + const events = [ + ev('server_start', { project_dir: '/p', version: '1.0.0', started_at: T(0) }, T(0)), + ev('lsp_event', { phase: 'ready', duration_ms: 250 }, T(0)), + ev('index_event', { index: 'schema', status: 'ready', queries: 5, mutations: 2 }, T(0)), + ev('tool_call', { + tool: 'validate_intent', duration_ms: 5, success: true, input: {}, + output: { ok: true, plan_id: 'p1', pending_files: ['app/views/pages/x.liquid'], pending_translations: ['k1'] }, + }, T(1)), + ev('tool_call', { + tool: 'enrich_error', duration_ms: 1, success: true, + input: { file_path: 'app/views/pages/x.liquid', check_name: 'MissingPartial' }, + }, T(2)), + ev('tool_call', { + tool: 'validate_code', duration_ms: 10, success: true, + input: { file_path: 'app/views/pages/x.liquid' }, + output: { errors: [{ check: 'MissingPartial' }], warnings: [], status: 'error' }, + }, T(3)), + ev('tool_call', { + tool: 'validate_code', duration_ms: 8, success: true, + input: { file_path: 'app/views/pages/x.liquid' }, + output: { errors: [], warnings: [], status: 'ok' }, + }, T(4)), + ev('tool_call', { + tool: 'scaffold', duration_ms: 100, success: true, + input: { write: true, model: 'note', type: 'crud' }, + output: { files: [{ path: 'a.liquid' }], written: ['a.liquid'] }, + }, T(5)), + ]; + + let incremental = initialState(); + for (const e of events) incremental = applyEvent(incremental, e); + + const bulk = replay(events); + expect(bulk).toEqual(incremental); + }); + + it('reducer is deterministic — same events twice yield equal states', () => { + const events = [ + ev('server_start', { project_dir: '/p', version: '1.0.0', started_at: T(0) }, T(0)), + ev('tool_call', { tool: 'lookup', duration_ms: 5, success: true }, T(0)), + ev('tool_call', { tool: 'lookup', duration_ms: 5, success: true }, T(0)), + ]; + expect(replay(events)).toEqual(replay(events)); + }); + + it('event count tracks both known and unknown kinds', () => { + const known = ev('tool_call', { tool: 'lookup', duration_ms: 5, success: true }); + const s = applyEvent(applyEvent(initialState(), known), known); + expect(s._event_count).toBe(2); + }); + + it('does not mutate the input state object', () => { + const s0 = initialState(); + const s0Snap = JSON.parse(JSON.stringify(s0)); + applyEvent(s0, ev('server_start', { project_dir: '/p', version: '1', started_at: T(0) })); + expect(s0).toEqual(s0Snap); + }); +}); diff --git a/tests/unit/structural-warnings.test.js b/tests/unit/structural-warnings.test.js index 9ace165..21a29ff 100644 --- a/tests/unit/structural-warnings.test.js +++ b/tests/unit/structural-warnings.test.js @@ -10,6 +10,7 @@ function getWarnings(content, filePath, existingChecks = new Set(), options = {} if (!ast) return []; const structural = extractAllFromAST(ast); const structuralObj = { + renders_used: structural.renders ?? [], tags_used: [...structural.tags], filters_used: [...structural.filters], doc_params: [...structural.docParams], @@ -56,6 +57,21 @@ describe('structural-warnings: HTML in pages', () => { expect(w.line).toBeGreaterThanOrEqual(0); expect(w.severity).toBe('warning'); }); + + // B-tier guard (2026-04-24): composite landing pages legitimately mix HTML + // wrappers with partial renders. Don't flag those — the check had 100% + // regression on exactly this pattern in the 2026-04-23 DEMO report. + it('does NOT warn for pages that render at least one partial (composite page)', () => { + const content = '---\nslug: index\n---\n
{% render "landing/hero" %}
'; + const warnings = getWarnings(content, '/project/app/views/pages/index.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:HtmlInPage')).toBe(false); + }); + + it('still warns for pure-HTML pages that do not render any partials', () => { + const content = '---\nslug: contact\n---\n
'; + const warnings = getWarnings(content, '/project/app/views/pages/contact.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:HtmlInPage')).toBe(true); + }); }); // ── GraphQL in partials ─────────────────────────────────────────────────── @@ -105,6 +121,71 @@ describe('structural-warnings: GraphQL in partials', () => { }); }); +// ── Multi-line graphql in {% liquid %} block ────────────────────────────── + +describe('structural-warnings: GraphqlMultilineInLiquidBlock', () => { + // Repro for the DEMO 2026-04-27 regression spiral. Multi-line `,` + // continuation inside `{% liquid %}` truncates the call; LSP fires + // GraphQLVariablesCheck.required for every dropped arg. The structural + // warning surfaces the syntactic root cause loudly, before the rule layer + // has to disambiguate. + it('errors on multi-line graphql with comma continuation inside {% liquid %} block', () => { + const content = + "{% liquid\n" + + "graphql result = 'contacts/create',\n" + + " name: shaped.name,\n" + + " email: shaped.email\n" + + "%}"; + const warnings = getWarnings(content, '/project/app/lib/commands/contacts/create.liquid'); + const w = warnings.find(w => w.check === 'pos-supervisor:GraphqlMultilineInLiquidBlock'); + expect(w).toBeDefined(); + expect(w.severity).toBe('error'); + expect(w.message).toContain('truncates'); + expect(w.message).toContain('single-line tag form'); + }); + + it('does NOT fire for the canonical {% graphql %} tag form', () => { + const content = "{% graphql result = 'op', name: shaped.name, email: shaped.email %}"; + const warnings = getWarnings(content, '/project/app/lib/commands/x.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:GraphqlMultilineInLiquidBlock')).toBe(false); + }); + + it('does NOT fire for single-line graphql inside {% liquid %} block', () => { + const content = + "{% liquid\n" + + "graphql result = 'op', name: shaped.name, email: shaped.email\n" + + "%}"; + const warnings = getWarnings(content, '/project/app/lib/commands/x.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:GraphqlMultilineInLiquidBlock')).toBe(false); + }); + + it('does NOT fire for multi-line graphql inside {% graphql %} tag delimiters', () => { + // `{%` … `%}` form parses multi-line correctly — only the {% liquid %} + // block continuation is truncated. + const content = + "{% graphql result = 'op',\n" + + " name: shaped.name,\n" + + " email: shaped.email %}"; + const warnings = getWarnings(content, '/project/app/lib/commands/x.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:GraphqlMultilineInLiquidBlock')).toBe(false); + }); + + it('reports each truncated call once per occurrence', () => { + const content = + "{% liquid\n" + + "graphql a = 'op_a',\n" + + " x: 1\n" + + "%}\n" + + "{% liquid\n" + + "graphql b = 'op_b',\n" + + " y: 2\n" + + "%}"; + const warnings = getWarnings(content, '/project/app/lib/commands/x.liquid'); + const found = warnings.filter(w => w.check === 'pos-supervisor:GraphqlMultilineInLiquidBlock'); + expect(found).toHaveLength(2); + }); +}); + // ── Shopify objects ──────────────────────────────────────────────────────── describe('structural-warnings: Shopify objects', () => { @@ -468,6 +549,54 @@ describe('structural-warnings: missing doc block', () => { // ── Method validation ───────────────────────────────────────────────────── +describe('structural-warnings: non-GET rendering page', () => { + // Landing-page mistake pattern the DEMO agent keeps repeating: + // `method: post` + HTML body → page 404s on browser GET. + it('warns when page has method: post and renders HTML content', () => { + const content = '---\nslug: contact\nmethod: post\nlayout: application\n---\n

Contact

\n
{{ foo }}
'; + const warnings = getWarnings(content, '/project/app/views/pages/contact.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:NonGetRenderingPage')).toBe(true); + }); + + it('warns when page has method: post and renders partials (composite landing)', () => { + const content = '---\nslug: index\nmethod: post\n---\n{% render "landing/hero" %}'; + const warnings = getWarnings(content, '/project/app/views/pages/index.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:NonGetRenderingPage')).toBe(true); + }); + + it('warns for put/delete/patch too', () => { + for (const method of ['put', 'delete', 'patch']) { + const content = `---\nslug: widget\nmethod: ${method}\n---\n
{{ x }}
`; + const warnings = getWarnings(content, '/project/app/views/pages/widget.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:NonGetRenderingPage')).toBe(true); + } + }); + + it('does NOT warn for API pages (slug under /api/)', () => { + const content = '---\nslug: api/contacts/create\nmethod: post\nformat: json\n---\n{{ r | json }}'; + const warnings = getWarnings(content, '/project/app/views/pages/api/contacts/create.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:NonGetRenderingPage')).toBe(false); + }); + + it('does NOT warn for JSON-only endpoints (no HTML, no layout, no partials, no output)', () => { + const content = '---\nslug: webhooks/stripe\nmethod: post\n---\n'; + const warnings = getWarnings(content, '/project/app/views/pages/webhooks/stripe.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:NonGetRenderingPage')).toBe(false); + }); + + it('does NOT warn for method: get', () => { + const content = '---\nslug: contact\nmethod: get\n---\n

Contact

'; + const warnings = getWarnings(content, '/project/app/views/pages/contact.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:NonGetRenderingPage')).toBe(false); + }); + + it('does NOT warn when method field is absent (default is get)', () => { + const content = '---\nslug: contact\n---\n

Contact

'; + const warnings = getWarnings(content, '/project/app/views/pages/contact.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:NonGetRenderingPage')).toBe(false); + }); +}); + describe('structural-warnings: method validation', () => { it('errors on uppercase POST', () => { const content = '---\nslug: test\nmethod: POST\n---\n{% assign x = 1 %}'; @@ -574,18 +703,33 @@ describe('structural-warnings: missing return in commands', () => { // ── Missing doc block in commands ───────────────────────────────────────── -describe('structural-warnings: missing doc block in commands', () => { - it('warns when command has no doc block', () => { +describe('structural-warnings: missing doc block in commands (B1.5 scope-out)', () => { + // Commands were dropped from MissingDocBlock after plan B1.5 — the check + // was 10% resolution / 40% regression on command files in production. + // Only partials still fire this warning. + it('does NOT warn when command has no doc block', () => { const content = '{% liquid\n assign object["id"] = 1\n return object\n%}'; const warnings = getWarnings(content, '/project/app/lib/commands/test/create.liquid'); - expect(warnings.some(w => w.check === 'pos-supervisor:MissingDocBlock')).toBe(true); + expect(warnings.some(w => w.check === 'pos-supervisor:MissingDocBlock')).toBe(false); }); - it('does not warn when command has doc block', () => { + it('does not warn when command has doc block either', () => { const content = '{% doc %}\n @param object {Hash}\n{% enddoc %}\n{% liquid\n return object\n%}'; const warnings = getWarnings(content, '/project/app/lib/commands/test/create.liquid'); expect(warnings.some(w => w.check === 'pos-supervisor:MissingDocBlock')).toBe(false); }); + + it('still warns for partials without doc block (regression guard)', () => { + const content = '
{{ x }}
'; + const warnings = getWarnings(content, '/project/app/views/partials/widget.html.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:MissingDocBlock')).toBe(true); + }); + + it('does not warn for queries without doc block', () => { + const content = '{% graphql res = "list_posts" %}{{ res | json }}'; + const warnings = getWarnings(content, '/project/app/lib/queries/list_posts.liquid'); + expect(warnings.some(w => w.check === 'pos-supervisor:MissingDocBlock')).toBe(false); + }); }); // ── Front matter key validation ─────────────────────────────────────────── diff --git a/tests/unit/translation-validator.test.js b/tests/unit/translation-validator.test.js new file mode 100644 index 0000000..c4957cb --- /dev/null +++ b/tests/unit/translation-validator.test.js @@ -0,0 +1,125 @@ +import { describe, it, expect } from 'bun:test'; +import { validateTranslationYaml } from '../../src/core/translation-validator.js'; + +function validate(content, filePath = 'app/translations/en.yml') { + return validateTranslationYaml(content, filePath); +} + +describe('translation-validator: missing top-level locale key', () => { + it('errors when the tree has no locale wrapper', () => { + const yaml = `app: + contact_form: + title: "Contact us" + success: "Thanks" +`; + const { errors } = validate(yaml); + expect(errors.some(e => e.check === 'pos-supervisor:TranslationMissingLocaleKey')).toBe(true); + expect(errors[0].message).toMatch(/en:/); + }); + + it('suggests the filename-based locale when available', () => { + const { errors } = validate('app:\n title: x\n', 'app/translations/de.yml'); + expect(errors[0].message).toMatch(/de:/); + }); + + it('falls back to en when filename is not a locale', () => { + const { errors } = validate('app:\n title: x\n', 'app/translations/strings.yml'); + expect(errors[0].message).toMatch(/en:/); + }); + + it('rejects multiple non-locale top-level keys', () => { + const yaml = `app: + title: x +ecommerce: + cart: y +`; + const { errors } = validate(yaml); + expect(errors).toHaveLength(1); + expect(errors[0].check).toBe('pos-supervisor:TranslationMissingLocaleKey'); + }); + + it('errors when top-level key looks like a typo locale (enff, enn, etc.)', () => { + const yaml = `enff: + app: + hello: "Hello" +`; + const { errors } = validate(yaml); + expect(errors).toHaveLength(1); + expect(errors[0].check).toBe('pos-supervisor:TranslationMissingLocaleKey'); + expect(errors[0].message).toMatch(/enff/); + }); +}); + +describe('translation-validator: valid locale wrappers', () => { + it('passes a correctly wrapped file', () => { + const yaml = `en: + app: + contact_form: + title: "Contact" +`; + const { errors, warnings } = validate(yaml); + expect(errors).toHaveLength(0); + expect(warnings).toHaveLength(0); + }); + + it('accepts two-letter locale with region (pt-BR)', () => { + const { errors } = validate('pt-BR:\n app:\n title: "x"\n', 'app/translations/pt-BR.yml'); + expect(errors).toHaveLength(0); + }); + + it('accepts a multi-locale file', () => { + const yaml = `en: + app: + title: "Contact" +de: + app: + title: "Kontakt" +`; + const { errors, warnings } = validate(yaml); + expect(errors).toHaveLength(0); + expect(warnings).toHaveLength(0); + }); +}); + +describe('translation-validator: mixed top-level keys', () => { + it('warns per stray non-locale key when at least one locale is present', () => { + const yaml = `en: + app: + title: "x" +app: + title: "y" +`; + const { errors, warnings } = validate(yaml); + expect(errors).toHaveLength(0); + expect(warnings.some(w => w.check === 'pos-supervisor:TranslationStrayTopKey' && w.message.includes('`app`'))).toBe(true); + }); +}); + +describe('translation-validator: edge cases', () => { + it('returns nothing on empty content', () => { + const { errors, warnings } = validate(''); + expect(errors).toHaveLength(0); + expect(warnings).toHaveLength(0); + }); + + it('returns nothing on comment-only content', () => { + const { errors, warnings } = validate('# just a comment\n'); + expect(errors).toHaveLength(0); + expect(warnings).toHaveLength(0); + }); + + it('errors on invalid YAML syntax', () => { + const { errors } = validate('en:\n app:\n title: "unterminated'); + expect(errors.some(e => e.check === 'pos-supervisor:TranslationYAML')).toBe(true); + }); + + it('errors when root is an array', () => { + const { errors } = validate('- one\n- two\n'); + expect(errors.some(e => e.check === 'pos-supervisor:TranslationStructure')).toBe(true); + }); + + it('errors when root is a scalar string', () => { + const { errors } = validate('just a string\n'); + expect(errors.some(e => e.check === 'pos-supervisor:TranslationStructure')).toBe(true); + }); +}); diff --git a/tests/unit/unknown-property-rules.test.js b/tests/unit/unknown-property-rules.test.js new file mode 100644 index 0000000..b3a99d9 --- /dev/null +++ b/tests/unit/unknown-property-rules.test.js @@ -0,0 +1,262 @@ +import { describe, it, expect, beforeEach } from 'bun:test'; +import { registerRules, clearRules, runRules } from '../../src/core/rules/engine.js'; +import { rules } from '../../src/core/rules/UnknownProperty.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; + +function buildGraphWithSchema() { + return buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, + graphql: {}, layouts: {}, translations: {}, assets: [], + schema: { + blog_post: { + path: 'app/schema/blog_post.yml', + properties: [ + { name: 'title' }, + { name: 'content' }, + { name: 'author' }, + { name: 'published_at' }, + { name: 'slug' }, + ], + }, + user_profile: { + path: 'app/schema/user_profile.yml', + properties: [ + { name: 'first_name' }, + { name: 'last_name' }, + { name: 'email' }, + { name: 'bio' }, + ], + }, + }, + }); +} + +function buildMinimalGraph() { + return buildFactGraph({ + pages: {}, partials: {}, commands: {}, queries: {}, + graphql: {}, schema: {}, layouts: {}, translations: {}, assets: [], + }); +} + +describe('UnknownProperty rules', () => { + beforeEach(() => { + clearRules(); + registerRules(rules); + }); + + describe('schema_property (priority 10)', () => { + it('matches when object is a known schema table', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'UnknownProperty', + params: { property: 'tittle', object: 'blog_post' }, + message: "Property 'tittle' does not exist on 'blog_post'", + file: 'app/views/pages/blog.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.schema_property'); + expect(result.confidence).toBe(0.85); + expect(result.hint_md).toContain('tittle'); + expect(result.hint_md).toContain('blog_post'); + expect(result.hint_md).toContain('title'); + }); + + it('matches plural form of schema name', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'UnknownProperty', + params: { property: 'emale', object: 'user_profiles' }, + message: "Property 'emale' does not exist on 'user_profiles'", + file: 'app/views/pages/users.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.schema_property'); + expect(result.hint_md).toContain('email'); + }); + + it('lists available properties', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'UnknownProperty', + params: { property: 'nonexistent', object: 'blog_post' }, + message: "Property 'nonexistent' does not exist on 'blog_post'", + file: 'app/views/pages/blog.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.hint_md).toContain('title'); + expect(result.hint_md).toContain('content'); + expect(result.hint_md).toContain('Available properties'); + }); + + it('does not match unknown schema table', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'UnknownProperty', + params: { property: 'foo', object: 'nonexistent_table' }, + message: "Property 'foo' does not exist on 'nonexistent_table'", + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).not.toBe('UnknownProperty.schema_property'); + }); + }); + + describe('context_property (priority 20)', () => { + it('matches when object starts with context and objectsIndex is loaded', () => { + const graph = buildMinimalGraph(); + const mockObjectsIndex = { + loaded: true, + contextObjects: () => [ + { handle: 'context.current_user', properties: ['id', 'email', 'first_name'] }, + { handle: 'context.session', properties: ['token', 'csrf'] }, + ], + }; + const diag = { + check: 'UnknownProperty', + params: { property: 'emai', object: 'context.current_user' }, + message: "Property 'emai' does not exist on 'context.current_user'", + file: 'app/views/pages/profile.liquid', + }; + const result = runRules(diag, { graph, objectsIndex: mockObjectsIndex }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.context_property'); + expect(result.confidence).toBe(0.7); + expect(result.hint_md).toContain('emai'); + expect(result.hint_md).toContain('email'); + expect(result.see_also).toBeDefined(); + expect(result.see_also.tool).toBe('domain_guide'); + }); + + it('does not match non-context objects', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'UnknownProperty', + params: { property: 'foo', object: 'some_var' }, + message: "Property 'foo' does not exist on 'some_var'", + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.generic'); + }); + }); + + describe('generic (priority 100)', () => { + it('matches any property/object pair as fallback', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'UnknownProperty', + params: { property: 'foo', object: 'bar' }, + message: "Property 'foo' does not exist on 'bar'", + file: 'app/views/pages/test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.generic'); + expect(result.confidence).toBe(0.4); + }); + + it('adds partial hint when file is in partials directory', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'UnknownProperty', + params: { property: 'foo', object: 'bar' }, + message: "Property 'foo' does not exist on 'bar'", + file: 'app/views/partials/my-partial.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.hint_md).toContain('doc'); + }); + + it('falls through to .default when params are missing', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'UnknownProperty', + params: {}, + message: 'Some unknown property error', + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.default'); + }); + }); + + describe('default catch-all (priority 1000)', () => { + it('does NOT preempt .schema_property', () => { + const graph = buildGraphWithSchema(); + const diag = { + check: 'UnknownProperty', + params: { property: 'tittle', object: 'blog_post' }, + message: "Property 'tittle' does not exist on 'blog_post'", + file: 'app/views/pages/blog.liquid', + }; + const result = runRules(diag, { graph }); + expect(result.rule_id).toBe('UnknownProperty.schema_property'); + }); + + it('does NOT preempt .context_property', () => { + const graph = buildMinimalGraph(); + const objectsIndex = { + loaded: true, + contextObjects: () => [{ handle: 'context.current_user', properties: ['id', 'email'] }], + }; + const diag = { + check: 'UnknownProperty', + params: { property: 'emai', object: 'context.current_user' }, + message: "Property 'emai' does not exist on 'context.current_user'", + file: 'test.liquid', + }; + const result = runRules(diag, { graph, objectsIndex }); + expect(result.rule_id).toBe('UnknownProperty.context_property'); + }); + + it('does NOT preempt .generic when both params are extracted', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'UnknownProperty', + params: { property: 'foo', object: 'bar' }, + message: "Property 'foo' does not exist on 'bar'", + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result.rule_id).toBe('UnknownProperty.generic'); + }); + + it('fires when only one of property/object was extracted', () => { + const graph = buildMinimalGraph(); + const diag = { + check: 'UnknownProperty', + params: { property: 'foo' }, // missing object + message: "Property 'foo' does not exist", + file: 'test.liquid', + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.default'); + }); + + it('fires when extraction failed entirely', () => { + const graph = buildMinimalGraph(); + const diag = { check: 'UnknownProperty' }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnknownProperty.default'); + expect(result.confidence).toBeLessThan(0.5); + }); + + it('hint covers typo / schema / partial-@param escape hatches', () => { + const graph = buildMinimalGraph(); + const diag = { check: 'UnknownProperty' }; + const result = runRules(diag, { graph }); + expect(result.hint_md).toContain('schema'); + expect(result.hint_md).toContain('@param'); + expect(result.hint_md).toContain('lookup'); + }); + }); +}); diff --git a/tests/unit/unused-assign-rules.test.js b/tests/unit/unused-assign-rules.test.js new file mode 100644 index 0000000..1a58fda --- /dev/null +++ b/tests/unit/unused-assign-rules.test.js @@ -0,0 +1,126 @@ +import { describe, it, expect, beforeEach } from 'bun:test'; +import { registerRules, clearRules, runRules, hasRules } from '../../src/core/rules/engine.js'; +import { rules } from '../../src/core/rules/UnusedAssign.js'; +import { buildFactGraph } from '../../src/core/project-fact-graph.js'; + +function buildGraph(overrides = {}) { + return buildFactGraph({ + pages: {}, + partials: { + 'blog_posts/list': { + path: 'app/views/partials/blog_posts/list.liquid', + params: ['page'], + renders: ['blog_posts/card'], + render_calls: [{ partial: 'blog_posts/card', args: ['blog_post'] }], + function_calls: [{ variable: 'items', path: 'queries/blog_posts/search' }], + rendered_by: [], + }, + 'blog_posts/card': { + path: 'app/views/partials/blog_posts/card.liquid', + params: ['blog_post'], + renders: [], + render_calls: [], + function_calls: [], + rendered_by: [], + }, + }, + commands: {}, + queries: { + 'app/lib/queries/blog_posts/search.liquid': { + params: ['query'], + graphql_calls: [], + function_calls: [], + }, + }, + graphql: {}, + schema: {}, + layouts: {}, + translations: {}, + assets: [], + ...overrides, + }); +} + +describe('UnusedAssign rules', () => { + beforeEach(() => { + clearRules(); + registerRules(rules); + }); + + it('registers rules for UnusedAssign', () => { + expect(hasRules('UnusedAssign')).toBe(true); + }); + + it('suppresses when variable is passed to render', () => { + const graph = buildGraph(); + const diag = { + check: 'UnusedAssign', + params: { variable: 'blog_post' }, + message: "'blog_post' is assigned but never used", + file: 'app/views/partials/blog_posts/list.liquid', + line: 5, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnusedAssign.passed_to_render'); + expect(result.confidence).toBe(0.95); + expect(result.suppress).toBe(true); + }); + + it('identifies function call result variable', () => { + const graph = buildGraph(); + const diag = { + check: 'UnusedAssign', + params: { variable: 'items' }, + message: "'items' is assigned but never used", + file: 'app/views/partials/blog_posts/list.liquid', + line: 3, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnusedAssign.passed_to_function'); + expect(result.confidence).toBe(0.8); + }); + + it('falls through to generic for truly unused variables', () => { + const graph = buildGraph(); + const diag = { + check: 'UnusedAssign', + params: { variable: 'totally_unused' }, + message: "'totally_unused' is assigned but never used", + file: 'app/views/partials/blog_posts/list.liquid', + line: 10, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnusedAssign.generic'); + expect(result.confidence).toBe(0.5); + }); + + it('handles missing params gracefully', () => { + const graph = buildGraph(); + const diag = { + check: 'UnusedAssign', + params: {}, + message: 'some weird message', + file: 'app/views/partials/blog_posts/list.liquid', + line: 1, + }; + const result = runRules(diag, { graph }); + expect(result).not.toBeNull(); + expect(result.rule_id).toBe('UnusedAssign.generic'); + }); + + it('handles file with no render or function calls', () => { + const graph = buildGraph(); + const diag = { + check: 'UnusedAssign', + params: { variable: 'unused' }, + message: "'unused' is assigned but never used", + file: 'app/views/partials/blog_posts/card.liquid', + line: 1, + }; + const result = runRules(diag, { graph }); + expect(result.rule_id).toBe('UnusedAssign.generic'); + }); +}); diff --git a/tests/unit/window-classifier.test.js b/tests/unit/window-classifier.test.js new file mode 100644 index 0000000..2a52014 --- /dev/null +++ b/tests/unit/window-classifier.test.js @@ -0,0 +1,431 @@ +import { describe, test, expect } from 'bun:test'; +import { + extractValidateCodeCalls, + classifyWindow, + classifyFixAdoption, + classifySession, + classifyWriteWindow, + extractWriteEvents, + buildEmitIndex, + computeCollateral, +} from '../../src/core/window-classifier.js'; + +function makeVcCall(overrides = {}) { + return { + kind: 'tool_call', + tool: 'validate_code', + success: true, + session_id: 'sess1', + ts: '2026-04-17T10:00:00Z', + input: { + file_path: 'app/views/pages/index.html.liquid', + content: '{% render "blog_posts/card" %}', + ...overrides.input, + }, + output: { + errors: [], + warnings: [], + ...overrides.output, + }, + ...overrides, + }; +} + +function makeDiag(check, message, line = 1) { + return { check, message, severity: 'error', line }; +} + +describe('extractValidateCodeCalls', () => { + test('groups calls by file', () => { + const events = [ + makeVcCall({ ts: 't1' }), + makeVcCall({ ts: 't2', input: { file_path: 'other.liquid', content: '' } }), + makeVcCall({ ts: 't3' }), + ]; + const byFile = extractValidateCodeCalls(events); + expect(byFile.size).toBe(2); + expect(byFile.get('app/views/pages/index.html.liquid')).toHaveLength(2); + expect(byFile.get('other.liquid')).toHaveLength(1); + }); + + test('skips failed calls', () => { + const events = [ + makeVcCall({ success: false }), + makeVcCall({ ts: 't2' }), + ]; + const byFile = extractValidateCodeCalls(events); + expect(byFile.get('app/views/pages/index.html.liquid')).toHaveLength(1); + }); + + test('skips non-validate_code events', () => { + const events = [ + { kind: 'tool_call', tool: 'scaffold', success: true, ts: 't1', input: {}, output: {} }, + makeVcCall(), + ]; + const byFile = extractValidateCodeCalls(events); + expect(byFile.size).toBe(1); + }); +}); + +describe('classifyWindow', () => { + test('resolved: diagnostic disappears between calls', () => { + const start = makeVcCall({ + output: { errors: [makeDiag('MissingPartial', "Missing partial 'blog_posts/card'")], warnings: [] }, + }); + const end = makeVcCall({ + ts: '2026-04-17T10:01:00Z', + output: { errors: [], warnings: [] }, + }); + const { window, outcomes } = classifyWindow(start, end); + expect(outcomes).toHaveLength(1); + expect(outcomes[0].outcome).toBe('resolved'); + expect(outcomes[0].check).toBe('MissingPartial'); + expect(window.file).toBe('app/views/pages/index.html.liquid'); + }); + + test('unchanged: diagnostic persists', () => { + const diag = makeDiag('MissingPartial', "Missing partial 'blog_posts/card'"); + const start = makeVcCall({ output: { errors: [diag], warnings: [] } }); + const end = makeVcCall({ ts: 't2', output: { errors: [diag], warnings: [] } }); + const { outcomes } = classifyWindow(start, end); + expect(outcomes).toHaveLength(1); + expect(outcomes[0].outcome).toBe('unchanged'); + }); + + test('moved: same template_fp but different fp (cross-file)', () => { + const start = makeVcCall({ + output: { errors: [makeDiag('MissingPartial', "Missing partial 'blog_posts/card'")], warnings: [] }, + }); + const end = makeVcCall({ + ts: 't2', + input: { file_path: 'app/views/pages/other.html.liquid', content: '' }, + output: { errors: [makeDiag('MissingPartial', "Missing partial 'blog_posts/card'")], warnings: [] }, + }); + const { outcomes } = classifyWindow(start, end); + expect(outcomes.some(o => o.outcome === 'moved')).toBe(true); + }); + + test('regressed: new diagnostic at end', () => { + const start = makeVcCall({ output: { errors: [], warnings: [] } }); + const end = makeVcCall({ + ts: 't2', + output: { errors: [makeDiag('UndefinedObject', "The object 'foo' is undefined")], warnings: [] }, + }); + const { outcomes } = classifyWindow(start, end); + expect(outcomes).toHaveLength(1); + expect(outcomes[0].outcome).toBe('regressed'); + }); + + test('mixed: one resolved, one unchanged, one regressed', () => { + const start = makeVcCall({ + output: { + errors: [ + makeDiag('MissingPartial', "Missing partial 'blog_posts/card'"), + makeDiag('UndefinedObject', "The object 'item' is undefined"), + ], + warnings: [], + }, + }); + const end = makeVcCall({ + ts: 't2', + output: { + errors: [ + makeDiag('UndefinedObject', "The object 'item' is undefined"), + makeDiag('UnknownFilter', "Unknown filter 'money'"), + ], + warnings: [], + }, + }); + const { outcomes } = classifyWindow(start, end); + + const resolved = outcomes.filter(o => o.outcome === 'resolved'); + const unchanged = outcomes.filter(o => o.outcome === 'unchanged'); + const regressed = outcomes.filter(o => o.outcome === 'regressed'); + + expect(resolved).toHaveLength(1); + expect(resolved[0].check).toBe('MissingPartial'); + expect(unchanged).toHaveLength(1); + expect(unchanged[0].check).toBe('UndefinedObject'); + expect(regressed).toHaveLength(1); + expect(regressed[0].check).toBe('UnknownFilter'); + }); + + test('window timestamps are correct', () => { + const start = makeVcCall({ ts: '2026-04-17T10:00:00Z', output: { errors: [], warnings: [] } }); + const end = makeVcCall({ ts: '2026-04-17T10:05:00Z', output: { errors: [], warnings: [] } }); + const { window } = classifyWindow(start, end); + expect(window.ts_start).toBe('2026-04-17T10:00:00Z'); + expect(window.ts_end).toBe('2026-04-17T10:05:00Z'); + expect(window.session_id).toBe('sess1'); + }); +}); + +describe('classifyFixAdoption', () => { + const mockBlobStore = { + getText(hash) { + if (hash === 'fix1') return 'View'; + return null; + }, + }; + + test('returns ignored when content unchanged', () => { + const content = '

Hello

'; + expect(classifyFixAdoption(content, content, [{ new_text_hash: 'fix1' }], mockBlobStore)).toBe('ignored'); + }); + + test('returns verbatim when fix text found in end content', () => { + const start = 'View'; + const end = 'View'; + const result = classifyFixAdoption(start, end, [{ new_text_hash: 'fix1' }], mockBlobStore); + expect(result).toBe('verbatim'); + }); + + test('returns partial when content changed but fix not applied verbatim', () => { + const start = 'View'; + const end = 'View'; + const result = classifyFixAdoption(start, end, [{ new_text_hash: 'fix1' }], mockBlobStore); + expect(result).toBe('partial'); + }); + + test('returns null when no proposed fixes', () => { + expect(classifyFixAdoption('a', 'b', [], mockBlobStore)).toBeNull(); + }); + + test('returns null when no blob store', () => { + expect(classifyFixAdoption('a', 'b', [{ new_text_hash: 'x' }], null)).toBeNull(); + }); +}); + +function makeWriteEvent(relPath, ts) { + return { kind: 'fs_watcher_sync', rel_path: relPath, ts }; +} + +describe('extractWriteEvents', () => { + test('groups by rel_path', () => { + const events = [ + makeWriteEvent('app/views/pages/a.liquid', 't1'), + makeWriteEvent('app/views/pages/b.liquid', 't2'), + makeWriteEvent('app/views/pages/a.liquid', 't3'), + ]; + const writes = extractWriteEvents(events); + expect(writes.size).toBe(2); + expect(writes.get('app/views/pages/a.liquid')).toHaveLength(2); + expect(writes.get('app/views/pages/b.liquid')).toHaveLength(1); + }); + + test('returns empty map for no write events', () => { + const events = [makeVcCall()]; + expect(extractWriteEvents(events).size).toBe(0); + }); + + test('ignores events without rel_path', () => { + const events = [{ kind: 'fs_watcher_sync', path: '/abs/path/file.liquid', ts: 't1' }]; + expect(extractWriteEvents(events).size).toBe(0); + }); + + test('sorts events chronologically per file', () => { + const events = [ + makeWriteEvent('app/views/pages/a.liquid', '2026-04-17T10:02:00Z'), + makeWriteEvent('app/views/pages/a.liquid', '2026-04-17T10:01:00Z'), + ]; + const writes = extractWriteEvents(events); + const sorted = writes.get('app/views/pages/a.liquid'); + expect(sorted[0].ts).toBe('2026-04-17T10:01:00Z'); + expect(sorted[1].ts).toBe('2026-04-17T10:02:00Z'); + }); +}); + +describe('classifyWriteWindow', () => { + const FILE = 'app/views/pages/index.html.liquid'; + + test('produces write_unverified outcomes for all diagnostics', () => { + const vc = makeVcCall({ + ts: '2026-04-17T10:00:00Z', + output: { + errors: [ + makeDiag('MissingPartial', "Missing partial 'blog_posts/card'"), + makeDiag('UndefinedObject', "The object 'item' is undefined"), + ], + warnings: [], + }, + }); + const write = makeWriteEvent(FILE, '2026-04-17T10:01:00Z'); + const { window, outcomes } = classifyWriteWindow(vc, write); + + expect(outcomes).toHaveLength(2); + expect(outcomes.every(o => o.outcome === 'write_unverified')).toBe(true); + expect(window.closed_by).toBe('write'); + expect(window.is_draft).toBe(false); + expect(window.ts_end).toBe('2026-04-17T10:01:00Z'); + expect(window.content_hash_end).toBeNull(); + }); + + test('produces empty outcomes when validation was clean', () => { + const vc = makeVcCall({ output: { errors: [], warnings: [] } }); + const write = makeWriteEvent(FILE, 't2'); + const { outcomes } = classifyWriteWindow(vc, write); + expect(outcomes).toHaveLength(0); + }); +}); + +describe('classifySession', () => { + const FILE = 'app/views/pages/index.html.liquid'; + + test('builds windows for files with multiple calls', () => { + const events = [ + makeVcCall({ + ts: 't1', + output: { errors: [makeDiag('MissingPartial', "Missing partial 'blog_posts/card'")], warnings: [] }, + }), + makeVcCall({ + ts: 't2', + output: { errors: [], warnings: [] }, + }), + ]; + const results = classifySession(events); + expect(results).toHaveLength(1); + expect(results[0].outcomes).toHaveLength(1); + expect(results[0].outcomes[0].outcome).toBe('resolved'); + }); + + test('skips files with only one call and no write event', () => { + const events = [makeVcCall()]; + const results = classifySession(events); + expect(results).toHaveLength(0); + }); + + test('creates N-1 windows for N calls to same file (no writes)', () => { + const events = [ + makeVcCall({ ts: 't1', output: { errors: [], warnings: [] } }), + makeVcCall({ ts: 't2', output: { errors: [], warnings: [] } }), + makeVcCall({ ts: 't3', output: { errors: [], warnings: [] } }), + ]; + const results = classifySession(events); + expect(results).toHaveLength(2); + expect(results[0].window.idx).toBe(0); + expect(results[1].window.idx).toBe(1); + }); + + test('validate-to-validate windows are tagged is_draft=1 when no write between them', () => { + const events = [ + makeVcCall({ ts: '2026-04-17T10:00:00Z', output: { errors: [], warnings: [] } }), + makeVcCall({ ts: '2026-04-17T10:01:00Z', output: { errors: [], warnings: [] } }), + ]; + const results = classifySession(events); + expect(results[0].window.is_draft).toBe(1); + expect(results[0].window.closed_by).toBe('validate'); + }); + + test('validate-to-validate windows are tagged is_draft=0 when write falls between them', () => { + const events = [ + makeVcCall({ ts: '2026-04-17T10:00:00Z', output: { errors: [], warnings: [] } }), + makeWriteEvent(FILE, '2026-04-17T10:00:30Z'), + makeVcCall({ ts: '2026-04-17T10:01:00Z', output: { errors: [], warnings: [] } }), + ]; + const results = classifySession(events); + // One validate-to-validate window, no write after last validate + expect(results).toHaveLength(1); + expect(results[0].window.is_draft).toBe(0); + }); + + test('creates write-closed window for validate then write (no re-validate)', () => { + const events = [ + makeVcCall({ + ts: '2026-04-17T10:00:00Z', + output: { errors: [makeDiag('MissingPartial', "Missing partial 'x'")], warnings: [] }, + }), + makeWriteEvent(FILE, '2026-04-17T10:01:00Z'), + ]; + const results = classifySession(events); + expect(results).toHaveLength(1); + expect(results[0].window.closed_by).toBe('write'); + expect(results[0].outcomes[0].outcome).toBe('write_unverified'); + }); + + test('creates both validate-to-validate and write-closed window for full multi-draft sequence', () => { + // Draft 1 → Draft 2 → write + const events = [ + makeVcCall({ + ts: '2026-04-17T10:00:00Z', + output: { errors: [makeDiag('MissingPartial', "Missing partial 'x'")], warnings: [] }, + }), + makeVcCall({ + ts: '2026-04-17T10:01:00Z', + output: { errors: [makeDiag('MissingPartial', "Missing partial 'x'")], warnings: [] }, + }), + makeWriteEvent(FILE, '2026-04-17T10:02:00Z'), + ]; + const results = classifySession(events); + expect(results).toHaveLength(2); + + const validateWindow = results.find(r => r.window.closed_by === 'validate'); + const writeWindow = results.find(r => r.window.closed_by === 'write'); + + expect(validateWindow).toBeDefined(); + expect(validateWindow.window.is_draft).toBe(1); // no write between the two validates + expect(writeWindow).toBeDefined(); + expect(writeWindow.outcomes[0].outcome).toBe('write_unverified'); + }); + + test('proper workflow: validate → write → re-validate produces non-draft window + no write-closed', () => { + // The ideal agent workflow + const events = [ + makeVcCall({ + ts: '2026-04-17T10:00:00Z', + output: { errors: [makeDiag('MissingPartial', "Missing partial 'x'")], warnings: [] }, + }), + makeWriteEvent(FILE, '2026-04-17T10:01:00Z'), + makeVcCall({ + ts: '2026-04-17T10:02:00Z', + output: { errors: [], warnings: [] }, + }), + ]; + const results = classifySession(events); + // One validate-to-validate window (write falls between → is_draft=0 → resolved) + // No write after last validate → no write-closed window + expect(results).toHaveLength(1); + expect(results[0].window.is_draft).toBe(0); + expect(results[0].window.closed_by).toBe('validate'); + expect(results[0].outcomes[0].outcome).toBe('resolved'); + }); +}); + +describe('buildEmitIndex', () => { + test('groups validator_emit events by fp', () => { + const events = [ + { kind: 'validator_emit', fp: 'a', proposed_fixes: [] }, + { kind: 'validator_emit', fp: 'a', proposed_fixes: [{ new_text_hash: 'x' }] }, + { kind: 'validator_emit', fp: 'b', proposed_fixes: [] }, + { kind: 'tool_call', tool: 'validate_code' }, + ]; + const index = buildEmitIndex(events); + expect(index.size).toBe(2); + expect(index.get('a')).toHaveLength(2); + expect(index.get('b')).toHaveLength(1); + }); +}); + +describe('computeCollateral', () => { + test('counts net new regressions', () => { + const outcomes = [ + { outcome: 'resolved' }, + { outcome: 'regressed' }, + { outcome: 'regressed' }, + { outcome: 'regressed' }, + ]; + expect(computeCollateral(outcomes)).toBe(2); + }); + + test('returns 0 when resolved >= regressed', () => { + const outcomes = [ + { outcome: 'resolved' }, + { outcome: 'resolved' }, + { outcome: 'regressed' }, + ]; + expect(computeCollateral(outcomes)).toBe(0); + }); + + test('returns 0 for all unchanged', () => { + expect(computeCollateral([{ outcome: 'unchanged' }, { outcome: 'unchanged' }])).toBe(0); + }); +}); diff --git a/tests/upstream/assign-syntax-coverage.test.js b/tests/upstream/assign-syntax-coverage.test.js new file mode 100644 index 0000000..c2cc023 --- /dev/null +++ b/tests/upstream/assign-syntax-coverage.test.js @@ -0,0 +1,116 @@ +/** + * Coverage probe for `InvalidAssignSyntax` and `InvalidOutputPush` — + * the assign/push grammar checks described in `docs/upstream-changes/ + * upstream-changes.md` sections A and C. + * + * Status as of pos-cli 6.0.7: these checks live in the parser-side repo + * but have NOT been shipped in `@platformos/platformos-check-common` yet. + * The parser dependency was bumped to ^0.0.17 in Phase 6 (the grammar + * change is in the parser); the LSP-level checks land separately. + * + * This file is a deferred coverage probe. It runs every CI cycle and + * LOGS whether the LSP now reports `InvalidAssignSyntax` / + * `InvalidOutputPush` for the canonical trigger inputs. It does NOT + * fail when the checks are absent — that's the current expected state. + * + * When the checks DO ship in a future pos-cli, the test surfaces it via + * the [SHIPPED] log line. At that point we should: + * 1. Add hint files (`src/data/hints/InvalidAssignSyntax.md`, + * `src/data/hints/InvalidOutputPush.md`). + * 2. Add rule modules (`src/core/rules/InvalidAssignSyntax.js`, + * `src/core/rules/InvalidOutputPush.js`) with priority-ordered + * categories, mirroring the Phase 3 ValidFrontmatter / JsonLiteralQuoteStyle + * treatment. + * 3. Add fingerprint pins in `tests/upstream/diagnostic-fingerprint.test.js`. + * 4. Replace the loose `expect(true).toBe(true)` with hard assertions on + * the emitted diagnostic shape (mirroring lsp-coverage-map.test.js). + * 5. Update `docs/upstream-changes/upstream-changes.md` to mark sections + * A and C as ABSORBED (currently OBSERVED-ONLY). + * + * The triggers below come straight from upstream PR-A and PR-C test cases + * so when the checks ship we'll already be aligned with their canonical + * inputs. + */ +import { describe, it, expect, beforeAll, afterAll, setDefaultTimeout } from 'bun:test'; +import { describePosCli } from '../integration/pos-cli/guard.js'; +import { startServer, FIXTURE_DIR } from '../integration/helpers/server.js'; + +setDefaultTimeout(30_000); + +let server; +beforeAll(async () => { server = await startServer(FIXTURE_DIR); }); +afterAll(() => server?.stop()); + +async function lspDiagsFor(filePath, content) { + const result = await server.callTool('validate_code', { file_path: filePath, content, mode: 'quick' }); + const all = [...result.errors, ...result.warnings]; + return all.filter(d => !d.check.startsWith('pos-supervisor:') && d.check !== 'OrphanedPartial'); +} + +function reportShipStatus(check, lspDiags) { + const matched = lspDiags.filter(d => d.check === check); + if (matched.length > 0) { + console.log(` [SHIPPED] ${check} — LSP fires the check. Action items in test header.`); + for (const d of matched.slice(0, 3)) { + console.log(` ${d.check} (${d.severity}): ${d.message?.slice(0, 100)}`); + } + } else { + console.log(` [PENDING] ${check} — still not shipped in this pos-cli. Triggers ready when it lands.`); + } +} + +describePosCli('Coverage probe: InvalidAssignSyntax (upstream PR-A, PR-C)', () => { + it('detects whether trailing-garbage assign syntax is flagged', async () => { + // Triggers from upstream `InvalidAssignSyntax.spec.ts`: stray `}` after a + // filter-array argument inside an assign. Tolerant parser folds back to + // string markup so other checks miss it; the dedicated check catches it. + const content = + "{% assign x = items | map: ['a', 'b'] } %}\n" + + "{% liquid\n assign y = items | join: ',' }\n%}\n"; + const lsp = await lspDiagsFor('app/views/partials/coverage_invalid_assign.liquid', content); + reportShipStatus('InvalidAssignSyntax', lsp); + // Always pass — this is a status probe, not a hard contract. + expect(true).toBe(true); + }); + + it('detects whether structurally-broken assign tags are flagged', async () => { + // Empty markup, missing operator, literal target, empty RHS. + const content = + "{% assign %}\n" + + "{% assign x %}\n" + + "{% assign 'literal' = 5 %}\n" + + "{% assign x = %}\n"; + const lsp = await lspDiagsFor('app/views/partials/coverage_assign_shapes.liquid', content); + reportShipStatus('InvalidAssignSyntax', lsp); + expect(true).toBe(true); + }); +}); + +describePosCli('Coverage probe: InvalidOutputPush (upstream PR-C)', () => { + it('detects whether `<<` in output position is flagged', async () => { + // The `<<` push operator is only valid inside `{% assign %}`. Using it + // in `{{ }}` or `{% echo %}` is invalid — upstream PR-C added a + // dedicated check for this. + const content = + "{{ items << 'x' }}\n" + + "{% echo items << 'x' %}\n"; + const lsp = await lspDiagsFor('app/views/partials/coverage_invalid_output_push.liquid', content); + reportShipStatus('InvalidOutputPush', lsp); + expect(true).toBe(true); + }); +}); + +describePosCli('Coverage probe: parser grammar — bare push form still parses', () => { + it('bare `{% assign arr << item %}` is the only valid push form post-PR-C', async () => { + // Grammar simplification kept the bare push form. This sanity check + // makes sure the parser doesn't regress on the canonical valid input. + const content = "{% assign arr = '' | split: '' %}\n{% assign arr << 'item' %}\n"; + const lsp = await lspDiagsFor('app/views/partials/coverage_bare_push.liquid', content); + const errs = lsp.filter(d => d.severity === 'error'); + if (errs.length > 0) { + console.log(' [REGRESSION CANDIDATE] bare push form produced errors — investigate:'); + for (const e of errs.slice(0, 3)) console.log(` ${e.check}: ${e.message}`); + } + expect(true).toBe(true); + }); +}); diff --git a/tests/upstream/diagnostic-fingerprint.test.js b/tests/upstream/diagnostic-fingerprint.test.js new file mode 100644 index 0000000..f3f751d --- /dev/null +++ b/tests/upstream/diagnostic-fingerprint.test.js @@ -0,0 +1,281 @@ +/** + * Diagnostic fingerprint stability — pins the masking algorithm + extracted + * params for known LSP / structural diagnostic messages. + * + * Why this is "upstream": fingerprints are the analytics-layer identity + * for a diagnostic. Anything that would change them — a tweak to the + * masking regex in diagnostic-record.js OR an upstream LSP message format + * change — would silently invalidate every dashboard scorecard that hangs + * off `template_fp`. This test makes such a change loud: + * - if YOU intentionally bumped the masking algorithm, regenerate the + * pinned fingerprints below and bump DIAGNOSTIC_RECORD_VERSION + * - if the LSP changed its message format, fix the extractor or template + * override before bumping (analytics history won't replay otherwise) + * + * Hashes are sha1 hex strings so they're deterministic across machines and + * Bun versions. + */ + +import { describe, it, expect } from 'bun:test'; +import { + templateOf, + templateFingerprint, + fingerprint, + extractParams, + makeDiagnosticRecord, +} from '../../src/core/diagnostic-record.js'; + +// One row per check we care about. `samples` is the list of distinct LSP +// messages we've observed; they MUST all collapse to the same template_fp. +// `expected_template` and `expected_template_fp` are the pinned values. +const FIXTURES = [ + { + check: 'MissingPartial', + samples: [ + "'forms/login' does not exist", + "'modules/core/_helpers' does not exist", + ], + expected_template: ' does not exist', + expected_template_fp: '40c4e1d01f5a7d9a533afd9dd30d5476d3a8f0e7', + expected_params: { partial: 'forms/login' }, + }, + { + check: 'UnknownFilter', + samples: [ + "Unknown filter 'json'", + "Unknown filter 'totally_made_up'", + ], + expected_template: 'Unknown filter ', + expected_template_fp: '8316c722fc77735b0c910d8e641a9738de614bde', + expected_params: { filter: 'json' }, + }, + { + check: 'UndefinedObject', + samples: [ + "The object 'product' is undefined", + "The object 'context_alias' is undefined", + ], + expected_template: 'The object is undefined', + expected_template_fp: '0c2d6c531571d7b992a04d376506355c49f8c06f', + expected_params: { variable: 'product' }, + }, + { + check: 'UnusedAssign', + samples: [ + "The variable 'x' is assigned but not used", + "The variable 'tmp_value' is assigned but not used", + ], + expected_template: 'The variable is assigned but not used', + expected_template_fp: '95c4bd44b70db0bd055b71bac359e9f43e657d13', + expected_params: { variable: 'x' }, + }, + { + check: 'TranslationKeyExists', + samples: [ + "Translation key 'foo.bar.baz' not found.", + "Translation key 'errors.login.invalid' not found.", + ], + expected_template: 'Translation key not found.', + expected_template_fp: '5e8202ce67ec71d879e74ff72794cdf39d86f1d9', + expected_params: { key: 'foo.bar.baz' }, + }, + { + check: 'MissingRenderPartialArguments', + samples: [ + "Missing required argument 'email' in render tag for partial 'sessions/form'", + "Missing required argument 'name' in render tag for partial 'users/profile'", + ], + expected_template: 'Missing required argument in render tag for partial ', + expected_template_fp: 'b11026d409d275c74cced1c558fe6bc551d8316c', + expected_params: { partial: 'sessions/form', missing_param: 'email' }, + }, + { + check: 'DeprecatedTag', + samples: [ + "Tag 'include' is deprecated, use 'render'", + "Tag 'parse_json' is deprecated, use 'assign'", + ], + expected_template: 'Tag is deprecated, use ', + expected_template_fp: 'e90d8c42f0463351bc1cf5f14375afabb43ff4fe', + expected_params: { tag: 'include', replacement: 'render' }, + }, + { + check: 'UnknownProperty', + samples: [ + "Unknown property `name` on `current_user`", + "Unknown property `slug` on `current_page`", + ], + expected_template: 'Unknown property on ', + expected_template_fp: '0da54aaa0a737429a152327283e50e664e899288', + expected_params: { property: 'name', object: 'current_user' }, + }, + { + check: 'GraphQLCheck', + samples: [ + 'Cannot query field "name" on type "Record"', + 'Cannot query field "slug" on type "Record"', + ], + expected_template: 'Cannot query field on type ', + expected_template_fp: '89868bdc426b9a6b4cb483e67bac42b10ab3ed86', + expected_params: { category: 'unknown_field_record', field: 'name', type: 'Record' }, + }, + + // ── ValidFrontmatter ─ pos-cli 6.0.7 multi-shape check ────────────────────── + // The check emits 8 distinct shapes. We pin one representative per shape so a + // template change in any shape is loud. The file-type label (Page/Form/etc.) + // is intentionally NOT masked — each (category, file_type) pair is a distinct + // analytics axis. The rule_id (set by the rule engine in core/rules/ + // ValidFrontmatter.js) groups across file types when needed. + { + check: 'ValidFrontmatter', + samples: [ + "Missing required frontmatter field 'name' in Form file", + "Missing required frontmatter field 'resource' in Form file", + ], + expected_template: 'Missing required frontmatter field in Form file', + expected_template_fp: '91177b77c72d510aedb42cd4d4e4dd275e2843c9', + expected_params: { category: 'missing_required', field: 'name', file_type: 'Form' }, + }, + { + check: 'ValidFrontmatter', + samples: [ + "Unknown frontmatter field 'cache' in Page file", + "Unknown frontmatter field 'title' in Page file", + ], + expected_template: 'Unknown frontmatter field in Page file', + expected_template_fp: '2461ba62fdf60c6335e84cf44e8f154134df30b5', + expected_params: { category: 'unknown_field', field: 'cache', file_type: 'Page' }, + }, + { + check: 'ValidFrontmatter', + samples: [ + "Layout 'application' does not exist", + "Layout 'modules/core/admin' does not exist", + ], + expected_template: 'Layout does not exist', + expected_template_fp: '20eb602a8f963006b5123f6e53975a17fdcbdd68', + expected_params: { category: 'layout_missing', layout: 'application' }, + }, + { + check: 'ValidFrontmatter', + samples: [ + "Invalid value 'POST' for 'method'. Must be one of: get, post, put, delete, patch", + "Invalid value 'PUT' for 'method'. Must be one of: get, post, put, delete, patch", + ], + expected_template: 'Invalid value for . Must be one of: get, post, put, delete, patch', + expected_template_fp: '055be883904858626da77da6c2a3436cabba3dfa', + expected_params: { + category: 'invalid_enum', + value: 'POST', + field: 'method', + allowed: 'get, post, put, delete, patch', + }, + }, + { + check: 'ValidFrontmatter', + samples: [ + "'layout_name' is deprecated", + "'layout_path' is deprecated", + ], + expected_template: ' is deprecated', + expected_template_fp: 'f34f5eb90a978fc0c05eccf2846ab86decedda7a', + expected_params: { category: 'deprecated_field', field: 'layout_name' }, + }, + { + check: 'ValidFrontmatter', + samples: [ + "Authorization policy 'guest_only' does not exist", + "Authorization policy 'admin_only' does not exist", + ], + expected_template: 'Authorization policy does not exist', + expected_template_fp: '21a6e70cef5fe5a0ac1c991425efd284504e7ecb', + expected_params: { category: 'association_missing', label: 'Authorization policy', name: 'guest_only' }, + }, + + // ── JsonLiteralQuoteStyle ─ single-shot constant message ──────────────────── + { + check: 'JsonLiteralQuoteStyle', + samples: [ + 'Use double quotes for string literals inside object/array literals (e.g. \'{"key": "value"}\', not "{\'key\': \'value\'}").', + ], + expected_template: 'Use double quotes for string literals inside object/array literals (e.g. , not ).', + expected_template_fp: '6330c359bab9fa907940ee6b41fc4f4ebef8dace', + expected_params: {}, + }, + + // ── DuplicateFunctionArguments ─ render and function variants ────────────── + { + check: 'DuplicateFunctionArguments', + samples: [ + "Duplicate argument 'foo' in function tag for partial 'helpers/can_do'.", + "Duplicate argument 'bar' in function tag for partial 'helpers/format'.", + ], + expected_template: 'Duplicate argument in function tag for partial .', + expected_template_fp: '02a96cb3b3010a94fac5eba0d545690f948184bb', + expected_params: { argument: 'foo', tag_kind: 'function', partial: 'helpers/can_do' }, + }, + { + check: 'DuplicateFunctionArguments', + samples: [ + "Duplicate argument 'name' in render tag for partial 'forms/login'.", + "Duplicate argument 'email' in render tag for partial 'forms/signup'.", + ], + expected_template: 'Duplicate argument in render tag for partial .', + expected_template_fp: '49a0e3b1f5cfdc41996898948d303ecc8fc9d710', + expected_params: { argument: 'name', tag_kind: 'render', partial: 'forms/login' }, + }, +]; + +// Structural (pos-supervisor:*) checks intentionally have NO mask (the tag +// name in "HTML element
in page" is part of the identity, not an +// identifier to collapse). Phase A2 doesn't add structural extractors — +// they fall through to {} and a single-instance fingerprint, which is +// the right behavior for now. + +describe('diagnostic-fingerprint: template + extractor pins', () => { + for (const fix of FIXTURES) { + describe(fix.check, () => { + it('first sample produces the pinned message_template', () => { + expect(templateOf(fix.check, fix.samples[0])).toBe(fix.expected_template); + }); + + it('all samples collapse to the same template_fp', () => { + const fps = fix.samples.map((m) => templateFingerprint(fix.check, templateOf(fix.check, m))); + expect(new Set(fps).size).toBe(1); + expect(fps[0]).toBe(fix.expected_template_fp); + }); + + it('first sample yields the pinned extracted params', () => { + expect(extractParams(fix.check, fix.samples[0])).toEqual(fix.expected_params); + }); + }); + } +}); + +describe('diagnostic-fingerprint: full record assembly', () => { + it('produces a deterministic fp for (check, file, template)', () => { + const r1 = makeDiagnosticRecord( + { check: 'MissingPartial', severity: 'error', message: "'a' does not exist", line: 0, column: 0 }, + { file: 'app/views/pages/x.liquid', source: 'lsp' }, + ); + const r2 = makeDiagnosticRecord( + { check: 'MissingPartial', severity: 'error', message: "'b' does not exist", line: 99, column: 99 }, + { file: 'app/views/pages/x.liquid', source: 'lsp' }, + ); + expect(r1.fp).toBe(r2.fp); + expect(r1.fp).toBe(fingerprint('MissingPartial', 'app/views/pages/x.liquid', ' does not exist')); + }); + + it('different files diverge fp but share template_fp', () => { + const r1 = makeDiagnosticRecord( + { check: 'MissingPartial', severity: 'error', message: "'a' does not exist" }, + { file: 'a.liquid', source: 'lsp' }, + ); + const r2 = makeDiagnosticRecord( + { check: 'MissingPartial', severity: 'error', message: "'a' does not exist" }, + { file: 'b.liquid', source: 'lsp' }, + ); + expect(r1.fp).not.toBe(r2.fp); + expect(r1.template_fp).toBe(r2.template_fp); + }); +});