add cli-to-js/plugin — automatic editor types via TS language-service by rayhanadev · Pull Request #2 · millionco/cli-to-js

rayhanadev · 2026-04-17T23:18:58Z

Summary

New subpath export `cli-to-js/plugin` registers a TypeScript language-service plugin that runs inside `tsserver`
Scans source for `convertCliToJs("")` / `fromHelpText("", ...)` calls, spawns each binary's `--help` in the background, parses it, and injects a module-augmented `KnownCliOptions` map so editors autocomplete flags per-binary with no codegen and no generic
Adds `KnownCliOptions` interface + specialized string-literal overloads to `convertCliToJs` / `fromHelpText` — loose default type remains when the argument isn't a literal

Usage

```jsonc
// tsconfig.json
{ "compilerOptions": { "plugins": [{ "name": "cli-to-js/plugin" }] } }
```

Config knobs: `disabled`, `timeout`, `helpFlag`, `allowList`, `denyList`. Env kill-switch: `CLI_TO_JS_PLUGIN_DISABLE=1`.

Caveats (documented in README)

Editor-only. `tsc` ignores language-service plugins — CI builds still need `--dts` or the generic.
String-literal binary names only; variables fall back to the loose default.
First paint is loose-typed until the async spawn resolves, then upgrades.
Plugin spawns binaries it discovers in source — use `allowList`/`denyList` to scope.

Test plan

`pnpm typecheck` passes
`pnpm build` produces `dist/plugin/index.cjs` with `module.exports = pluginInit` (required by tsserver's factory check) and `index.d.cts` with `export = pluginInit`
`pnpm check` (lint + fmt) clean
Existing suite green (327 tests)
Manual smoke: add the plugin to a consumer `tsconfig.json`, write `const git = await convertCliToJs("git")`, confirm `git.commit({...})` autocompletes `message`/`amend`/etc. and rejects unknown keys

🤖 Generated with Claude Code

…ic editor types Registers a tsserver plugin that scans source for convertCliToJs("<literal>") and fromHelpText("<literal>", ...) calls, spawns each binary's --help in the background, and injects a module-augmented KnownCliOptions map so editors autocomplete subcommand flags without any codegen or generic. Editor-only — tsc ignores language-service plugins. For CI builds, keep using --dts or the <T> generic. Configurable via tsconfig plugin options (disabled, timeout, helpFlag, allowList, denyList) and CLI_TO_JS_PLUGIN_DISABLE env var. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Adds an editor-only TypeScript language-service plugin for cli-to-js that discovers convertCliToJs("...") / fromHelpText("...") call sites, runs <bin> --help in the background, parses the output, and injects KnownCliOptions types via a virtual .d.ts so editors can provide per-binary flag autocomplete without codegen or user-supplied generics.

Changes:

Adds cli-to-js/plugin subpath export and a new plugin build output (CJS + .d.cts types).
Introduces plugin implementation: scanning TS program, resolving schemas by spawning binaries, and emitting module augmentation.
Extends public typings with KnownCliOptions and literal-name overloads for convertCliToJs / fromHelpText.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
pnpm-lock.yaml	Updates lockfile to include TypeScript in the root importer dev deps.
packages/cli-to-js/vite.config.ts	Switches to multi-pack build to output both ESM library and CJS plugin bundle.
packages/cli-to-js/src/plugin/scan.ts	Implements AST scanning for relevant calls with string-literal binary names.
packages/cli-to-js/src/plugin/resolve.ts	Spawns `<bin> --help`, selects best output, parses into `CliSchema`.
packages/cli-to-js/src/plugin/index.ts	Registers tsserver plugin, maintains cache, serves a virtual `.d.ts`, and triggers rescans/resolutions.
packages/cli-to-js/src/plugin/emit.ts	Emits module augmentation for `KnownCliOptions` based on parsed schema.
packages/cli-to-js/src/index.ts	Adds `KnownCliOptions` and overload-based typing for literal binary names.
packages/cli-to-js/src/constants.ts	Adds plugin-specific timeout/debounce constants.
packages/cli-to-js/package.json	Adds `./plugin` export and TypeScript peer/dev dependency metadata.
README.md	Documents how to enable/configure the TypeScript plugin and its limitations.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-17T23:24:27Z

+      const projectDirectory = info.project.getCurrentDirectory();
+      const virtualFilePath = path.resolve(projectDirectory, VIRTUAL_FILE_NAME);
+
+      const binaryCache = new Map<string, BinaryCacheEntry>();
+      const inflightResolutions = new Map<string, Promise<void>>();
+      let virtualFileContent = buildCurrentContent(binaryCache);
+      let virtualFileVersion = 0;
+      let lastScanSignature = "";
+      let debounceHandle: NodeJS.Timeout | null = null;
+
+      const host = info.languageServiceHost;
+
+      const originalGetScriptFileNames = host.getScriptFileNames.bind(host);
+      host.getScriptFileNames = () => {
+        const existing = originalGetScriptFileNames();
+        if (existing.includes(virtualFilePath)) return existing;
+        return [...existing, virtualFilePath];
+      };
+
+      const originalGetScriptSnapshot = host.getScriptSnapshot.bind(host);
+      host.getScriptSnapshot = (fileName) => {
+        if (fileName === virtualFilePath) {
+          return tsModule.ScriptSnapshot.fromString(virtualFileContent);
+        }
+        return originalGetScriptSnapshot(fileName);
+      };
+
+      const originalGetScriptVersion = host.getScriptVersion.bind(host);
+      host.getScriptVersion = (fileName) => {
+        if (fileName === virtualFilePath) return String(virtualFileVersion);
+        return originalGetScriptVersion(fileName);
+      };
+
+      const originalGetScriptKind = host.getScriptKind?.bind(host);
+      if (originalGetScriptKind) {
+        host.getScriptKind = (fileName) => {
+          if (fileName === virtualFilePath) return tsModule.ScriptKind.TS;
+          return originalGetScriptKind(fileName);


virtualFilePath is built with path.resolve(...) and later compared with fileName === virtualFilePath in multiple host hooks. Depending on tsserver host/path normalization (especially on Windows: separator/casing), these exact-string comparisons can fail, causing the virtual file snapshot/version/kind not to be served. It’s safer to normalize/canonicalize paths using TypeScript’s path utilities (e.g., tsModule.server.toNormalizedPath / info.project.projectService.toCanonicalFileName) and compare canonical forms consistently.

Copilot · 2026-04-17T23:24:27Z

+const TARGET_CALL_NAMES = new Set(["convertCliToJs", "fromHelpText"]);
+const NODE_MODULES_SEGMENT = "/node_modules/";
+
+const isTargetCallee = (expression: LeftHandSideExpression, tsModule: TsModule): boolean => {
+  if (tsModule.isIdentifier(expression)) {
+    return TARGET_CALL_NAMES.has(expression.text);
+  }
+  if (tsModule.isPropertyAccessExpression(expression)) {
+    return TARGET_CALL_NAMES.has(expression.name.text);
+  }
+  return false;


The scan matches calls purely by callee name (convertCliToJs / fromHelpText) and does not verify that the symbol actually comes from the cli-to-js module. This can produce false positives (e.g., a local helper with the same name or an unrelated .convertCliToJs(...) method) and trigger unintended binary spawns. Consider using the type-checker to resolve the call signature/symbol and confirm it originates from an import of cli-to-js before enqueueing a resolution.

Copilot · 2026-04-17T23:24:27Z

+const NODE_MODULES_SEGMENT = "/node_modules/";
+
+const isTargetCallee = (expression: LeftHandSideExpression, tsModule: TsModule): boolean => {
+  if (tsModule.isIdentifier(expression)) {
+    return TARGET_CALL_NAMES.has(expression.text);
+  }
+  if (tsModule.isPropertyAccessExpression(expression)) {
+    return TARGET_CALL_NAMES.has(expression.name.text);
+  }
+  return false;
+};
+
+export const scanCliCalls = (program: Program, tsModule: TsModule): DiscoveredCall[] => {
+  const discoveredCalls: DiscoveredCall[] = [];
+
+  for (const sourceFile of program.getSourceFiles()) {
+    if (sourceFile.isDeclarationFile) continue;
+    if (sourceFile.fileName.includes(NODE_MODULES_SEGMENT)) continue;
+


NODE_MODULES_SEGMENT is hard-coded as "/node_modules/", so sourceFile.fileName.includes(...) won’t filter dependency paths that use backslashes (common on Windows). That can cause the plugin to scan files under node_modules and potentially spawn binaries based on dependency source. Consider a separator-agnostic check (e.g., normalize to POSIX, or check for both /node_modules/ and \\node_modules\\).

Copilot · 2026-04-17T23:24:27Z

+const escapeStringLiteral = (value: string): string => `"${value.replace(/["\\]/g, "\\$&")}"`;
+
+const propertyKey = (name: string): string => {
+  if (/^[A-Za-z_$][A-Za-z0-9_$]*$/.test(name)) return name;
+  return escapeStringLiteral(name);
+};


escapeStringLiteral only escapes " and \\, but does not escape newlines, carriage returns, tabs, or other characters that can make the emitted .d.ts invalid. Since this plugin consumes arbitrary --help output (including choices), prefer a more complete escaping strategy (e.g., using JSON.stringify(value) for string literal emission) to ensure valid TypeScript in all cases.

Copilot · 2026-04-17T23:24:28Z

+import { convertCliToJs } from "cli-to-js";
+
+const git = await convertCliToJs("git"); // ← plugin augments the return type
+await git.commit({ message: "hi" }); //   ← flags autocomplete, unknown keys error


The README claims the plugin “rejects unknown keys error”, but the public types currently allow arbitrary option keys via an index signature ([key: string]: unknown) in SubcommandFn/SpawnFn, so unknown properties won’t be a type error (you’ll still get autocomplete for known flags). Either adjust the wording here to avoid promising unknown-key rejection, or tighten the option types to remove the permissive index signature when using KnownCliOptions.

Suggested change

await git.commit({ message: "hi" }); // ← flags autocomplete, unknown keys error

await git.commit({ message: "hi" }); // ← known flags autocomplete with typed option shapes

Copilot · 2026-04-17T23:24:28Z

+      const scanAndQueue = (): void => {
+        const program = info.languageService.getProgram();
+        if (!program) return;
+        const calls = scanCliCalls(program, tsModule);
+        const unique = new Set<string>();
+        for (const call of calls) {
+          if (isBinaryPermitted(call.binaryName)) {
+            unique.add(call.binaryName);
+          }
+        }
+
+        const signature = [...unique].sort().join("\u0000");
+        if (signature === lastScanSignature) return;
+        lastScanSignature = signature;
+
+        for (const binaryName of unique) {
+          if (!binaryCache.has(binaryName)) {
+            beginResolution(binaryName);
+          }
+        }
+      };
+
+      const scheduleScan = (): void => {
+        if (debounceHandle) return;
+        debounceHandle = setTimeout(() => {
+          debounceHandle = null;
+          try {
+            scanAndQueue();
+          } catch (scanError) {
+            info.project.projectService.logger.info(
+              `cli-to-js/plugin scan failed: ${String(scanError)}`,
+            );
+          }
+        }, PLUGIN_REGENERATE_DEBOUNCE_MS);
+      };


New TypeScript-language-service plugin logic (scan → resolve → emit virtual .d.ts) is introduced without automated tests. Since this package already has a Jest/Vitest-style suite, consider adding unit tests for scanCliCalls (literal detection + node_modules filtering), emitAugmentation (escaping + module augmentation shape), and the cache update behavior in the plugin entrypoint (including removal of no-longer-present binaries).

Copilot · 2026-04-17T23:24:28Z

+      const scanAndQueue = (): void => {
+        const program = info.languageService.getProgram();
+        if (!program) return;
+        const calls = scanCliCalls(program, tsModule);
+        const unique = new Set<string>();
+        for (const call of calls) {
+          if (isBinaryPermitted(call.binaryName)) {
+            unique.add(call.binaryName);
+          }
+        }
+
+        const signature = [...unique].sort().join("\u0000");
+        if (signature === lastScanSignature) return;
+        lastScanSignature = signature;
+
+        for (const binaryName of unique) {
+          if (!binaryCache.has(binaryName)) {
+            beginResolution(binaryName);
+          }
+        }
+      };


scanAndQueue never prunes binaryCache entries for binaries that are no longer discovered in the latest scan. That means the virtual augmentation can keep stale KnownCliOptions entries (and editor completions) even after the last convertCliToJs("...") / fromHelpText("...") call is removed from the workspace. Consider removing cache entries that are not present in the newly computed unique set (and cancelling/ignoring any inflight resolution for them) before rebuilding the virtual file content.

…o-js/plugin tsserver loads plugins with its own legacy resolver — it does not honor package.json "exports" and only looks at plugin/package.json, plugin/index.js, etc. With "type": "module" at the root, a plain plugin/index.js also fails because it's treated as ESM. Adds: - plugin/index.js CJS shim re-exporting the built bundle - plugin/index.d.ts ambient type for the factory - plugin/package.json scopes the shim directory to "type": "commonjs" Verified: "Plugin validation succeeded" in tsserver.log against a scratch consumer project. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ubcommands Root cause of the Debug Failure in setDocument: adding a path to getScriptFileNames while only faking fileExists/getScriptSnapshot leaves the projectService's script-info cache empty. When createProgram walks the root files it calls Debug.checkDefined(getScriptInfoForPath(...)) and crashes the whole tsserver process. Fix: write the generated augmentations.d.ts to a real file in <tmpdir>/cli-to-js-plugin/<hash>/, so TS's normal file-loading path creates the ScriptInfo. Removes all the host overrides except getScriptFileNames. Also pipes subcommand enrichment (enrichSubcommands) into the resolver so each subcommand surfaces real per-flag types instead of Record<string, unknown>. Verified against /tmp/cli-to-js-plugin-test: "Plugin validation succeeded", no crash, augmentations.d.ts grows to ~12 KB with real per-subcommand option shapes (e.g. git.clone: { verbose?: boolean; quiet?: boolean; ... }). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

SubcommandFn<TOptions> was intersecting TOptions with { [key: string]: unknown }, which let any typo pass. The intersection also leaked into the ConvertCliToJs overload via `KnownCliOptions[N] & Record<string, Record<string, unknown>>`, defeating strict property checking even with the explicit generic from the README. Dropping the index signature from SubcommandFn/SpawnFn keeps the loose default (TOptions defaults to Record<string, unknown>, which is itself an index signature) while enabling excess-property checks when TOptions is specific. The overload now uses a conditional to preserve the strict shape from KnownCliOptions without the widening intersection. Verified in /tmp/cli-to-js-plugin-test: the plugin augmentation now flags `git.commit({ messaeg: "oops" })` as a typo with a did-you-mean suggestion. All 327 existing tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Two parallel agents informed this turn: Agent A surfaced chrF (Popović 2015, WMT) as the canonical published reference for what my 50/50 Jaccard-plus-char-3-gram blend was approximating; Agent B argued the WIDTH problem ("does a new player reach guess 2?") gates the DEPTH problem ("does the per-guess feedback help?"). Shipping both width and depth this turn — they're complementary primitives. 1) chrF replaces the blend (canonical primitive, ten-year-old reference). The 50/50 Jaccard + char-3-gram-cosine I shipped in v1.1 was an ad-hoc invention. chrF is the published peer-reviewed metric for exactly this problem: a multi-n character-n-gram F-score, used as the no-model baseline in sacrebleu and WMT. - chrfSimilarity(player, target): n = 1..6, F-β with β = 2 (recall-weighted; rewards a player whose output covers the target's content even with extra material — matches the "many prompts land here" spirit of the game). Pure JS, ~30 LOC, O(n). - describe() switches to chrfSimilarity. Thresholds unchanged (0.6 / 0.4 / 0.2). - blendedSimilarity and charNgramSimilarity are deleted. Per the patch-camouflage and dedup bans: no fallback path, no parallel metrics, no deprecation shim. A single owner of "similarity" at the scorer's owning layer. chrF is inherently asymmetric under β = 2 (recall-weighted on target-as-reference). That asymmetry is correct and intentional; the test that previously asserted symmetry was wrong and is replaced with one that asserts asymmetry under β=2. Reference: Popović, "chrF: character n-gram F-score for automatic MT evaluation" (WMT 2015). 2) Per-guess token diff (Wordle-grade feedback). Until now each guess showed only a similarity % + bucket label + length-delta. A new player had no idea WHY their guess scored where it did, so couldn't improve — they were playing in the dark. Wordle's defining feature is the per-letter colour map, not the daily mechanic alone; Inversion's equivalent is per-token diff between the player's live output and the target output. - diffOutputs(player, target) → { shared, onlyInTarget, onlyInPlayer } in scorer.ts. Pure, deterministic, order-stable. - After each guess, cli.ts prints: ✓ shared: the tokens you matched + missed: target tokens you didn't produce − extra: your tokens that aren't in target - Common function words (the, a, of, in, …) are filtered from display only — they still count toward the chrF score. List cap at 10 tokens per row with "+N more" overflow. STOP_WORDS lives in constants.ts; DIFF_DISPLAY_CAP next to it. Display-layer filtering keeps the metric pure. 3) `inversion tutorial` subcommand (cold-start cliff). Agent B's #1 ranked primitive: a new player burns their UTC-day daily on confusion if they don't know to run `inversion practice <id>` first. The tutorial is a zero-stakes guided round. - TUTORIAL_PUZZLE: a separate hand-cached puzzle ("Name three fruits, comma-separated, no extra text." → "Apple, Banana, Mango") generated via the same sandboxed claude call that bundled puzzles use, so the live output stays in the same neighbourhood as the cached one. NOT in BUNDLED_PUZZLES — never appears in the daily rotation. - cmdTutorial(): prints a 7-line intro explaining the mechanic, then runs playSession against TUTORIAL_PUZZLE with recordResult = false (no streak effect). - `inversion tutorial` is added to the help banner as the first subcommand listed, so a fresh user sees it before `play`. Deferred from Agent B: scripted pre-filled first guess + post- guess narration ("here's why that scored where it did"). v1.3 ships the minimum viable tutorial — a separate puzzle marked tutorial — without the scripted walkthrough. Engineering discipline observed: - Archaeology gate: chrF replaces the blend at the scorer's owning layer; nothing in callers changed. Token diff goes next to chrF in the same module — same domain (similarity), same boundary. Tutorial reuses playSession verbatim with recordResult = false; no new state machine. - Invariant gate: similarity ∈ [0,1] preserved; identical → 1; fully-disjoint → 0; deterministic given the two outputs. - Patch-camouflage / cargo-culting / dedup bans: deleted blendedSimilarity and charNgramSimilarity rather than keeping them as deprecated re-exports. Deleted the 50/50 blend constants. Tutorial does not introduce a separate state machine — reuses playSession. - Deletion pass: blendedSimilarity, charNgramSimilarity (and their constants), and the cosineOfCounts helper are all gone. Verification: - pnpm typecheck clean. - pnpm build clean (dist/cli.mjs 14.6 kB / gzip 4.3 kB). - pnpm test 69/69 across 4 files (+5 chrF tests, +6 diffOutputs tests, +2 tutorial-puzzle invariant tests; symmetry test rewritten as asymmetry test under β=2). - pnpm check clean (74 files, 0 lint warnings). - Live `claude` smoke: - inversion tutorial with a fair guess ("List three fruits separated by commas.") → live output "Apple, banana, mango." → chrF 100 % / convergent / 🟩 / solved 1/6. Diff row: "✓ shared: apple, banana, mango". Share grid emitted. - No repo-context leakage (runner sandbox from v1.2 holding). Deferred (with criterion): - Scripted pre-filled first guess + narrated feedback in the tutorial — *unclear product behaviour at v1*; the minimum viable tutorial may already be enough. Re-evaluate after first-run feedback. - :reveal mid-session escape (Agent B's millionco#2) — *missing demand signal*; players can already exit on EOF. - A free structural-hint primitive ("format: list" / "length: short") inspired by escape-room "free clue" cards (Agent B's unknown-unknown probe) — *defer to v2*.

Copilot AI review requested due to automatic review settings April 17, 2026 23:18

Copilot started reviewing on behalf of rayhanadev April 17, 2026 23:19 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

rayhanadev marked this pull request as draft April 17, 2026 23:33

rayhanadev and others added 2 commits April 17, 2026 19:37

ethan-wickstrom mentioned this pull request May 13, 2026

inversion v1.4: :reveal escape + chrF + token diff + tutorial ethan-wickstrom/clijs#1

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add cli-to-js/plugin — automatic editor types via TS language-service#2

add cli-to-js/plugin — automatic editor types via TS language-service#2
rayhanadev wants to merge 4 commits into
mainfrom
ray/feat-ts-plugin

rayhanadev commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	await git.commit({ message: "hi" }); // ← flags autocomplete, unknown keys error
	await git.commit({ message: "hi" }); // ← known flags autocomplete with typed option shapes

Conversation

rayhanadev commented Apr 17, 2026

Summary

Usage

Caveats (documented in README)

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants