Skip to content

WORKLOG.md

Latest commit

 

History

History
265 lines (200 loc) · 15.6 KB

WORKLOG.md

File metadata and controls

265 lines (200 loc) · 15.6 KB

WORKLOG

Running log of substantive work on this repo. Newest entries on top. Each entry captures: what was done, what was found, what was decided, what's still open.

This is a working document — short, dense, written for the next agent (human or AI) picking up where the last session left off.

2026-06-11 — AI-7072 plugin-side mitigation: considered, rejected

Question. With AI-7072 surfaced by Phase B, should the plugin ship a defensive guard that snapshots dbt_packages/* sha256 before+after altimate_dbt_build, and prepend a WARNING: to the tool output if any hashes drift?

Decision: no. Drop the workaround; the right fix lives upstream in altimate-dbt-integration.

Why

The altimate plugin surface is being built fresh in three places at once: this opencode plugin, a hermes plugin, and the existing Claude Code plugin. Shipping a plugin-side detector here means either (a) duplicating the same workaround into each plugin, or (b) leaving the other plugins silent on the same bug. Both are worse than fixing it once upstream where all three benefit. The greenfield state of the plugin code means we can hold the line against half-measures without legacy compatibility cost.

Verification we ran on the candidate detector (artifacts under .scratch/runs/2026-06-11__14-3{1,3}-*__phase-B*/):

Scenario Detector behavior
Edit file content → build ✅ WARNING fires, names the file
Clean build, no edits ✅ no false-positive WARNING
Delete a file → build; deps recreates with identical content ❌ silent — before/after hashes match, no diff, user's deletion was undone with no signal

So even the detector is partial: it catches "content-change clobber" but misses "deletion-then-restoration", which is the same class of upstream side effect. Half-measure caught on the way out.

What was kept

  • AI-7072 (Jira, project AI, Bug, Priority High, assignee Haider). Real upstream fix tracked there.
  • TESTING.md Phase B as-is: strong pass / fail criteria. No new "soft pass via warning" tier — that tier would only exist to validate the workaround we decided not to ship.
  • TESTING_RESULTS.md Phase B as-is: continues to report the silent revert as a fail.

What was reverted

All changes from the candidate fix/AI-7072-dbt-packages-clobber-warning branch were reverted before any commit: snapshotDbtPackages / diffPackageSnapshots / formatPackageClobberWarning helpers and their wiring into altimateDbtBuild, the new TESTING.md "soft pass" tier, the TESTING_RESULTS.md rewrite of the Phase B detail, and the phase-b.sh verdict update that recognised the WARNING tier. The branch itself stayed local; nothing was pushed.

Principle for the next session

If a phase reveals an upstream bug with a tracked ticket and a known real fix, do not ship a plugin-side detector / warning / shim from this repo for it — even one that's codex-clean and runs green on the repro. The plugin tree is greenfield and we have leverage in three plugin repos at once; the right fix is upstream, and the cost of shipping the workaround is duplicated maintenance, partial coverage (see the deletion-restore hole above), and users assuming the bug is handled.

The exception is a bug in this plugin's own code (manifest, tool defs, wrappers, schemas). Those still get patched here.

2026-06-11 — Phases A–D (agent-as-decider, dbt_packages, stdin-wedge, cross-warehouse)

Goal. Re-validate TESTING_RESULTS.md against the headline failure modes from plugin-skill-experiments/ (especially 03-issues-and-fixes.md Issue #13 and 05-ade-bench-experiment.md Runs 5–6). Phases 1–6 prove the tools wire up correctly; A–D probe whether the agent actually uses them in realistic flows and whether the plugin exposes upstream side effects.

Outcome

Phase Verdict Anchor run dir
A — agent-as-decider ⚠️ soft (3/3 variants picked a plugin tool — altimate_dbt_columns, not altimate_code — over Read/Bash/grep; the strict "called altimate_code" criterion was not met) .scratch/runs/2026-06-11__02-55-55__phase-A__{bare,softnudge,mandatory}/
B — in-flight dbt_packages/* edit protection ❌ fail — Issue #13 reproduced .scratch/runs/2026-06-11__02-58-02__phase-B/
C — stdin-wedge guard reality check ✅ pass (guard not load-bearing in altimate-code 0.8.3; kept anyway) .scratch/runs/2026-06-11__03-00-06__phase-C/
D — cross-warehouse smoke ⚠️ inconclusive on strict criteria (plugin shim drove altimate-code into both warehouses; MSSQL connector failed → only Snowflake side was actually compared) .scratch/runs/2026-06-11__03-01-26__phase-D/

Tightened verdict language in TESTING_RESULTS.md: "all phases pass" now means wiring works AND the agent reliably selects the plugin tool when prompted realistically. Phases 1–6 alone (force-invoking via opencode debug agent --tool ...) are insufficient.

Plugin-side fixes shipped this round

  1. ToolContext.directory instead of process.cwd() for the default cwd of all 5 tools (commit b6fc209). Surfaced under Phase A: the agent did invoke altimate_code, but the spawned subprocess ran in a stale path from opencode's runtime cwd instead of the dbt project the host session opened in. Phase 5's earlier passes had only worked because we were running opencode debug agent from inside the project dir directly. Once the driver moved up to the repo root, the bug surfaced.
  2. stdin-wedge guard comment refreshed to record the Phase C re-validation against altimate-code 0.8.3. Stays as belt-and-suspenders.

Findings that are not plugin-side and were NOT patched

  • Issue #13 (dbt_packages/* clobber) — root cause is in altimate-dbt-integration/src/dbtIntegrationAdapter.ts:390-408 + configuration.ts:41. Reported in TESTING_RESULTS.md Phase B with the full sha trail. Recommendation: flip installDepsOnProjectInitialization: true → false upstream, OR add a --no-deps flag on bundled altimate-dbt schema-verify / build. Per the protocol, no upstream code was modified from this repo.
  • opencode default model selection picked google/gemini-3-pro-image-preview (no tool support) when both Anthropic and OpenRouter providers were configured → hard fail before any agent turn. Worked around by forcing --model anthropic/claude-haiku-4-5. Not a plugin bug.
  • MSSQL connection failure in Phase D — env-side (Connection 'eastman_source_mssql': FAILED, consistent with Issue #4 / Issue #7 in 03-issues-and-fixes.md). Snowflake side worked. Plugin shim did its job.

Fixture changes

  • Bootstrapped ~/.local/share/opencode/auth.json from altimate-code's auth.json so opencode has direct Anthropic creds. Called out here so future green results aren't mistaken for "the plugin made opencode auth work."
  • No edits to ~/.dbt/profiles.yml — used the existing eastman profiles as-is.
  • Airbnb fixture under .scratch/integration/ untouched between phases (the Phase B edit was restored at the end of the script).

Open follow-ups

  • Phase A only covered the "deterministic plugin tool covers the question" case. A follow-up phase should ask the agent something that requires multi-step work (no altimate_dbt_* match) and verify it picks altimate_code rather than grep+read.
  • Phase B has no good plugin-side mitigation; ship the upstream PR.
  • Phase D didn't end-to-end the cross-warehouse comparison because of the MSSQL connector failure. A follow-up should pre-bake the MSSQL adapter and retest, OR pick two warehouses that don't need extra adapter setup (e.g. two Snowflake accounts, or Snowflake + BigQuery).
  • The stdin-wedge guard comment now claims altimate-code 0.8.3. If the installed version rolls forward, re-run Phase C and update the comment (or drop the guard entirely if you decide the belt-and-suspenders cost isn't worth it).

Commits

SHA Subject
b6fc209 fix: use ToolContext.directory as default cwd instead of process.cwd()
(this round, to come) docs: TESTING.md Phases A–D + TESTING_RESULTS.md + comment refresh

2026-06-10 / 2026-06-11 — v1.17 port + TESTING.md execution

Goal. Execute the 6-phase validation plan in TESTING.md end-to-end against the scaffolded plugin.

Outcome

All 6 phases pass on fix/opencode-plugin-v1.17 (3 commits ahead of main, unpushed).

Phase Test Result
1 bun run typecheck Pass after porting plugin from @opencode-ai/plugin v1.2.20 → v1.17
2 Plugin discovery via opencode debug config Pass (scope: local)
3 Skill discovery via opencode debug skill Pass — all 11 altimate skills load
4 Tool registration via opencode debug agent build Pass — all 5 altimate_* tools register
5 Each tool end-to-end via opencode debug agent ... --tool ... against airbnb DuckDB fixture Pass after subcommand fixes
6 Failure modes (missing binaries, bad model name) Pass — all paths return ERROR:-prefixed strings

Bugs found in the scaffold

  1. Plugin code targeted @opencode-ai/plugin v1.2.20, runtime resolves v1.17.1. Three breaking changes between those versions:

    • tool({parameters: z.object(...)}) → tool({args: <ZodRawShape>})
    • Plugin = {name, tools: {...}} object → Plugin = async (input) => ({tool: {...}}) function returning Hooks
    • Return type: declared ToolResult accepts {output, metadata}, but the host wrapper at altimate-code/packages/opencode/src/tool/registry.ts:163-176 overwrites metadata with truncation info and shoves the full plugin return into its own output, which session/message-v2.ts:toModelOutput then tries to read as {text, attachments}. Net effect: plugin metadata is dropped before the model sees it. The only safe return shape is a plain string. Errors encoded as ERROR: ... prefixed strings.

  2. opencode.json schema was wrong on two keys — silently ignored by opencode (which hard-fails on invalid config but the wrong keys were unknown, not invalid):

    • plugins (plural) → schema only defines plugin (singular)
    • skills: ["./path", ...] → schema expects skills: {paths: ["./path", ...]}

  3. altimate_dbt_source called the wrong subcommand. Scaffold used source; altimate-dbt's actual CLI subcommand is columns-source. The source subcommand silently printed help and exited 0 — no error, just wrong output.

  4. altimate_dbt_build used the wrong selector flag. Scaffold passed --select (dbt-style); altimate-dbt only accepts --model <name> [--downstream]. Empirically verified: altimate-dbt build --select dim_listings silently builds the entire project (Found 12 models). Switched to --model, added a downstream arg, kept select as a back-compat alias that routes to --model. Explicit error when downstream: true without a model so a targeted build can't silently fan out to the full project.

  5. altimate_delegate tool name was the odd one out. Renamed to altimate_code to match the rest of the surface (skill name altimate-code, binary altimate-code, file plugins/altimate-code/index.ts, sibling tools altimate_dbt_*).

Prerequisites that were missing

  • opencode CLI not installed → npm install -g opencode-ai@1.17.1
  • No global ~/.config/opencode/opencode.json → created one pointing at the plugin

Operational changes

  • Moved integration scratch dir /tmp/altimate-opencode-integration → .scratch/integration/
  • Added .scratch/ and .github/meta/ to .gitignore
  • Updated TESTING.md Phase 5 + 6 expectations to match the v1.17 string-return contract (errors are ERROR:-prefixed strings, not {error: ...} JSON objects)
  • Updated README.md altimate_dbt_build signature

Commits on fix/opencode-plugin-v1.17

SHA Date Subject
dd4c79a 2026-06-11 refactor: rename altimate_delegate → altimate_code
ebf8e5c 2026-06-10 chore: ignore .scratch/ + .github/meta/, move integration out of /tmp
36d7764 2026-06-10 fix: port plugin to @opencode-ai/plugin v1.17 and fix CLI subcommands

Codex review iterations

7 rounds total. Findings adopted: 4. Findings rejected: 3 (all rejections documented in commit messages).

Rejected findings, for the record:

  • R5 "preserve --select for back-compat". Empirically verified --select was silently ignored by altimate-dbt — there was no working behavior to preserve.
  • R6 "don't flatten JSON into a string". Contradicts R1's own finding (which we'd already empirically confirmed against the host wrapper code). v1.17 plugin tools must return strings; the JSON is preserved verbatim inside the string.
  • R7 "rename of altimate_delegate is a breaking change". Plugin is 0.1.0, unpublished, no remote, previous scaffold didn't typecheck — no existing consumers.

When codex flip-flops between contradictory findings across rounds, stop the loop and make the call rather than burning rounds.

Phase 5 detail — what each tool actually returned

Tool Args Outcome
altimate_dbt_columns {"model":"dim_listings"} String ending in \n[]\n. The [] is what altimate-dbt's columns actually returns from this fixture — models/dim/dim.yml only documents 2 of the model's columns, manifest state was inconsistent. Plugin shim correct; TESTING.md's expected LISTING_ID, LISTING_NAME, ... doesn't match what the fixture's schema.yml actually produces.
altimate_dbt_source {"source":"airbnb","table":"hosts"} 5083-char string ending in []. models/sources.yml has no columns: block on any source — empty array is expected.
altimate_dbt_compile {"model":"dim_listings_hosts"} 5893-char string containing select, from, dim_listings, dim_hosts. Matches TESTING.md exactly.
altimate_dbt_build {"model":"dim_listings"} After the --select → --model fix: builds dim_listings only (no dim_hosts mentions, expected fct_reviews mentions from test relationships). Back-compat: {"select":"dim_listings"} produces identical behavior.
altimate_code {"task":"list the columns of the dim_listings model and exit","timeout_sec":300} 6076-char string. Contains altimate-code's Performing one time database migration first-run banner + LLM-synthesized markdown bullet list of all 8 columns + raw altimate-dbt JSON output from altimate-code's internal tool calls. Confirmed end-to-end agent delegation works.

Known gaps in Phase 5 testing

altimate_code (the headline feature) was only exercised with one trivial task. Not verified:

  • Multi-step delegation (chained altimate-code tool calls)
  • Cross-warehouse work (the actual README justification)
  • Timeout behavior — does timeout_sec cleanly kill long-running subprocesses?
  • stdin-wedge guard — stdio: ["ignore", ...] is kept as a guard against an upstream bug, but we never confirmed reproduction with stdin: "inherit" to prove the guard works
  • Output truncation behavior on large altimate-code responses
  • --continue / session resume
  • --dir override correctness

Notes for the next session

  • Branch fix/opencode-plugin-v1.17 is unpushed; no remote configured.
  • Global config at ~/.config/opencode/opencode.json registers this plugin by absolute path. If the repo moves, that path needs updating.
  • ~/.config/opencode/plugins/altimate is a symlink to this repo.
  • DuckDB-backed dbt fixture at .scratch/integration/ is gitignored — re-seed from any small dbt project if you nuke .scratch/. The validation used an internal "airbnb" fixture not redistributed with this repo.