Skip to content

[issues/557] Automate 33 assisted integration tests via closeQuickOpen dismissal#561

Merged
couimet merged 7 commits into
mainfrom
issues/557
May 13, 2026
Merged

[issues/557] Automate 33 assisted integration tests via closeQuickOpen dismissal#561
couimet merged 7 commits into
mainfrom
issues/557

Conversation

@couimet
Copy link
Copy Markdown
Owner

@couimet couimet commented May 13, 2026

Summary

workbench.action.closeQuickOpen can programmatically dismiss VS Code QuickPicks and InputBoxes from tests — the human was only needed to press Escape, not to select items. This converts 33 [assisted] tests to fully automated across 9 test files, reducing the assisted count from 147 to 112 and increasing automated from 86 to 119. Also removes the now-dead { assisted: true } option from standardSuite since it only printed a cosmetic banner — test filtering is handled by the [assisted] prefix in test names.

Changes

  • Added openAndDismiss(command) helper that fires a command, settles, dismisses via closeQuickOpen, and awaits the promise — a one-liner replacing every waitForHuman Escape prompt
  • Converted 33 [assisted] tests to automated across 9 files: terminalPicker.test.ts (12), filePicker.test.ts (8), statusBarMenu.test.ts (4), coreSendCommands.test.ts (3), customAiAssistants.test.ts (2), bindToDestination.test.ts (1), editorBindingValidation.test.ts (1), clipboardPreservation.test.ts (1), goToRangeLink.test.ts (1)
  • Removed StandardSuiteOptions interface and assisted parameter from standardSuite — all 33 call sites are now two-argument form
  • Removed printAssistedBanner function from assistedTestHelper.ts (dead code after option removal)
  • Updated TESTING.md to document the closeQuickOpen dismissal pattern and openAndDismiss helper
  • Flipped 33 QA YAML entries from automated: assisted to automated: true

Test Plan

  • All 1874 unit tests pass (98.51% coverage)
  • QA validator passes (119 automated / 112 assisted / 4 false)
  • test:release:automated — 161 of 173 automated tests pass (12 pre-existing flaky failures in smartPadding, textEditorDestination, sendFilePath — unrelated to this change)

Related

Summary by CodeRabbit

  • Tests

    • Enabled programmatic dismissal of pickers/input dialogs to fully automate open→capture→dismiss flows.
    • Converted many formerly assisted integration tests to fully automated execution.
    • Added a standardized open-and-dismiss helper and simplified test harness for consistent test patterns.
    • Added automated coverage for a built-in assistant’s destination-picker scenario.
  • Documentation

    • Updated testing guidance and QA rules to reflect expanded automation and clarify when assisted mode is still required.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 13, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0cfad428-4841-49e5-93f0-10b09168dfbb

📥 Commits

Reviewing files that changed from the base of the PR and between 1089ad6 and 72793b9.

📒 Files selected for processing (1)
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/rangelink-vscode-extension/src/integration-tests/helpers/testEnv.ts

Walkthrough

Adds openAndDismiss(command), simplifies the standardSuite API (removes assisted/banner), documents programmatic QuickPick/InputBox dismissal, updates QA YAML entries, and converts many integration tests from assisted human steps to programmatic open→log→dismiss flows.

Changes

Test Automation Refactor

Layer / File(s) Summary
Helper refactor & openAndDismiss
packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts, packages/rangelink-vscode-extension/src/__integration-tests__/helpers/standardSuite.ts, packages/rangelink-vscode-extension/src/__integration-tests__/helpers/index.ts, packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts
Add exported openAndDismiss(command): Promise<void>, simplify standardSuite(name, fn) by removing StandardSuiteOptions/assisted and banner printing, remove printAssistedBanner export, and update helper barrel exports.
Documentation & QA metadata
packages/rangelink-vscode-extension/TESTING.md, packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yaml, CLAUDE.md
Document automated QuickPick/InputBox dismissal via workbench.action.closeQuickOpen, update guidance for assisted-only scenarios, and flip many QA entries from assisted to true.
Built-in AI Assistants — Destination Picker
packages/rangelink-vscode-extension/src/__integration-tests__/suite/builtInAiAssistants.test.ts
Add a new non-assisted suite that opens the destination picker, extracts logged QuickPick items, and asserts “GitHub Copilot Chat” appears as bindable; remove printAssistedBanner() calls.
Status Bar Menu
packages/rangelink-vscode-extension/src/__integration-tests__/suite/statusBarMenu.test.ts
Replace waitForHuman with openAndDismiss(CMD_OPEN_STATUS_BAR_MENU), remove assisted suite param, and assert QuickPick log structure for bound/unbound states.
Terminal Picker
packages/rangelink-vscode-extension/src/__integration-tests__/suite/terminalPicker.test.ts
Import command IDs and openAndDismiss, remove assisted suite param, and convert multiple picker-dismissal tests to openAndDismiss(...).
File Picker
packages/rangelink-vscode-extension/src/__integration-tests__/suite/filePicker.test.ts
Import command IDs and openAndDismiss, remove assisted suite param, and convert multiple picker-dismissal tests to openAndDismiss(...).
Core Send Commands
packages/rangelink-vscode-extension/src/__integration-tests__/suite/coreSendCommands.test.ts
Expand command imports, add openAndDismiss, remove assisted suite param, and convert “no bound destination” tests to openAndDismiss(...).
Custom AI Assistants
packages/rangelink-vscode-extension/src/__integration-tests__/suite/customAiAssistants.test.ts
Import CMD_BIND_TO_DESTINATION and openAndDismiss, remove assisted suite param, and replace waitForHuman flows with openAndDismiss(...).
Bind / Clipboard / Editor-binding / Go-to
multiple src/__integration-tests__/suite/* files
Convert focused suites (bindToDestination, clipboardPreservation, editorBindingValidation, goToRangeLink) to use openAndDismiss(CMD_*) and remove assisted harness wiring.
Mechanical standardSuite signature updates
many src/__integration-tests__/suite/* files
Remove empty options objects and { assisted: true } from standardSuite calls across many test files; no test logic changes.

Sequence Diagram

sequenceDiagram
  participant TestRunner
  participant VSCode
  participant Extension
  TestRunner->>VSCode: executeCommand(CMD_...) // start command promise
  TestRunner->>TestRunner: settle() // wait for logs to appear
  loop while command pending
    TestRunner->>VSCode: executeCommand('workbench.action.closeQuickOpen') // dismiss QuickPick/InputBox
  end
  VSCode->>Extension: command invoked (handler runs)
  VSCode->>TestRunner: command promise resolves
  TestRunner->>TestRunner: settle() // final wait for logs
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Possibly related PRs

Poem

"🐰
I opened, peeked, then gently closed the view,
commands did the work — no extra hands were due.
Logs held the hints, assertions hopped along,
CI blinks green — the rabbit sings its song.
🥕"

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main objective: converting 33 assisted tests to automated via closeQuickOpen dismissal, matching the primary change across the PR.
Linked Issues check ✅ Passed The PR successfully addresses all coding objectives from issue #557: implements openAndDismiss helper, converts 33 tests from assisted to automated, removes StandardSuiteOptions/printAssistedBanner, updates TESTING.md and QA YAML, and validates the pattern across multiple test files.
Out of Scope Changes check ✅ Passed All changes are directly scoped to automating integration tests: new openAndDismiss helper, test file updates removing assisted flags, helper refactoring, documentation updates, and QA metadata. No unrelated code modifications detected.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issues/557

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 13, 2026

⚠️ QA Coverage Gap Detected

The PR updates the automation status of several test cases from 'assisted' to 'automated', indicating that they can now be fully automated. However, there may still be gaps in coverage for scenarios that require manual interaction.

Suggested test cases:

  • R-M Status Bar Menu: Verify that the QuickPick menu opens correctly when clicking the status bar item. (automatable)
  • R-M Status Bar Menu: Verify that the QuickPick menu opens correctly when using the keybinding. (automatable)
  • R-M Status Bar Menu: Check that the menu contains the correct items after opening. (automatable)

Generated by QA Gap Check (GPT-4o-mini via GitHub Models)

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts (1)

7-7: ⚡ Quick win

Restore a named width constant for the section divider.

Inlining 60 reintroduces a magic number and makes future width tuning brittle.

♻️ Proposed fix
-const SECTION_LINE = '─'.repeat(60);
+const SECTION_LINE_WIDTH = 60;
+const SECTION_LINE = '─'.repeat(SECTION_LINE_WIDTH);

As per coding guidelines, "Do not use magic numbers—define named constants for all numeric literals with semantic meaning using SCREAMING_SNAKE_CASE".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts`
at line 7, Restore a named width constant for the section divider: introduce a
SCREAMING_SNAKE_CASE constant (e.g., SECTION_WIDTH) and use it in the existing
SECTION_LINE definition (SECTION_LINE = '─'.repeat(SECTION_WIDTH)) instead of
inlining 60; update any related references or tests that assumed the numeric
literal so future width tuning uses the single constant.
packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts (1)

310-314: ⚡ Quick win

Extract selection bounds into named constants.

Line 311–313 uses semantic numeric literals directly (0, 2). Please define local SCREAMING_SNAKE_CASE constants for these bounds to keep intent explicit and consistent.

Proposed refactor
+    const SELECTION_START_LINE = 0;
+    const SELECTION_START_COLUMN = 0;
+    const SELECTION_END_LINE = 2;
+    const SELECTION_END_COLUMN = 0;
     const editor009 = await openEditor(fileUri);
     editor009.selection = new vscode.Selection(
-      new vscode.Position(0, 0),
-      new vscode.Position(2, 0),
+      new vscode.Position(SELECTION_START_LINE, SELECTION_START_COLUMN),
+      new vscode.Position(SELECTION_END_LINE, SELECTION_END_COLUMN),
     );

As per coding guidelines, **/*.{ts,tsx,js,jsx}: “Do not use magic numbers—define named constants for all numeric literals with semantic meaning using SCREAMING_SNAKE_CASE”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts`
around lines 310 - 314, The selection uses inline numeric literals when
constructing vscode.Selection/vscode.Position (in the test where
editor009.selection is set); extract these magic numbers into descriptive
SCREAMING_SNAKE_CASE constants (e.g. START_LINE, START_COLUMN, END_LINE,
END_COLUMN) at the top of the test or just above the selection setup and then
use those constants when creating the two vscode.Position instances and the
vscode.Selection so the intent of 0 and 2 is explicit and follows the project's
no-magic-numbers guideline.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts`:
- Around line 37-41: The openAndDismiss helper can hang because a single await
settle() + one close action races against the command promise; change
openAndDismiss to repeatedly attempt dismissal until the command promise
resolves (or a short timeout) by: call vscode.commands.executeCommand(command)
into promise, then loop: await settle(), invoke
vscode.commands.executeCommand('workbench.action.closeQuickOpen') (or a no-op if
not open), check if promise settled (use Promise.race with a small timeout or
inspect resolution), and break when the command completes; reference
openAndDismiss, settle and the 'workbench.action.closeQuickOpen' command to
locate and update the implementation.

In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/editorBindingValidation.test.ts`:
- Line 20: The import order is wrong: move the "import assert from
'node:assert'" statement to the group of Node built-in imports so it appears
before any project or third-party imports to satisfy the lint/import-order rule;
locate the existing "import assert from 'node:assert'" line in the test file and
place it with the other built-ins (top of the import block) so built-ins precede
project imports.

In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/statusBarMenu.test.ts`:
- Around line 33-38: The test names "status-bar-menu-002" and the similar one at
lines 67-78 claim to verify clicking the status bar item / Cmd+R Cmd+M
keybinding but they only call CMD_OPEN_STATUS_BAR_MENU via openAndDismiss;
either update the tests to actually exercise the UI/keybinding paths or change
their titles/assertions to reflect they invoke the command directly. Locate the
tests by the identifiers 'status-bar-menu-002' (and the second case around lines
67-78), and either (a) replace the openAndDismiss(CMD_OPEN_STATUS_BAR_MENU) call
with a real UI interaction such as clicking the status bar item helper (e.g.,
clickStatusBarItem or the existing helper that targets the status bar) or
simulating the keybinding sequence (e.g., sendKeybinding for Cmd+R then Cmd+M),
or (b) rename the test descriptions to indicate they are invoking the command
directly and adjust assertions accordingly so coverage claims match actual
behavior.

In `@packages/rangelink-vscode-extension/TESTING.md`:
- Line 78: Update the broken reference in CLAUDE.md that still points to
"QuickPick limitation": change the text at the referenced location to read "See
TESTING.md § \"QuickPick and InputBox dismissal\" for what can and cannot be
automated" or update any direct anchor link to use
`#quickpick-and-inputbox-dismissal`; ensure the section title/name in TESTING.md
("QuickPick and InputBox dismissal") is used exactly so the cross-reference
resolves.

---

Nitpick comments:
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts`:
- Line 7: Restore a named width constant for the section divider: introduce a
SCREAMING_SNAKE_CASE constant (e.g., SECTION_WIDTH) and use it in the existing
SECTION_LINE definition (SECTION_LINE = '─'.repeat(SECTION_WIDTH)) instead of
inlining 60; update any related references or tests that assumed the numeric
literal so future width tuning uses the single constant.

In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts`:
- Around line 310-314: The selection uses inline numeric literals when
constructing vscode.Selection/vscode.Position (in the test where
editor009.selection is set); extract these magic numbers into descriptive
SCREAMING_SNAKE_CASE constants (e.g. START_LINE, START_COLUMN, END_LINE,
END_COLUMN) at the top of the test or just above the selection setup and then
use those constants when creating the two vscode.Position instances and the
vscode.Selection so the intent of 0 and 2 is explicit and follows the project's
no-magic-numbers guideline.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c398d930-b65f-4304-b210-846638e5bb7f

📥 Commits

Reviewing files that changed from the base of the PR and between 243e9f0 and 05e1f0e.

📒 Files selected for processing (34)
  • packages/rangelink-vscode-extension/TESTING.md
  • packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yaml
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/index.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/standardSuite.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/bindToDestination.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/commandRegistration.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorContent.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorTab.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuExplorer.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuTerminal.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/coreSendCommands.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/customAiAssistants.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/dirtyBufferWarning.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/editorBindingValidation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/filePathNavigation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/filePicker.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/filenameOnlyNavigation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/goToRangeLink.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/linkGeneration.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationClamping.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationPrecision.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationToastSettings.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/platformKeybindings.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/releaseNotifier.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/sendFilePath.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/smartPadding.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/statusBarMenu.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/terminalPicker.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/textEditorDestination.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/unbind.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/untitledNavigation.test.ts

Comment thread packages/rangelink-vscode-extension/TESTING.md
couimet added a commit that referenced this pull request May 13, 2026
…umber fixes, stale references

Added retry loop to openAndDismiss so the helper doesn't race against slow picker rendering on loaded CI. Extracted magic numbers into named constants per P003. Updated stale cross-references and test descriptions to reflect the switch from UI triggers to direct command invocation.

Ignored Feedback:
- Import order in editorBindingValidation.test.ts: False positive — assert is already at line 1 before all other imports.

Ref: #561 (review)
couimet added 5 commits May 13, 2026 13:31
…en` dismissal

## Summary

`workbench.action.closeQuickOpen` can programmatically dismiss VS Code QuickPicks and InputBoxes from tests — the human was only needed to press Escape, not to select items. This converts 33 `[assisted]` tests to fully automated across 9 test files, reducing the assisted count from 147 to 112 and increasing automated from 86 to 119. Also removes the now-dead `{ assisted: true }` option from `standardSuite` since it only printed a cosmetic banner — test filtering is handled by the `[assisted]` prefix in test names.

## Changes

- Added `openAndDismiss(command)` helper that fires a command, settles, dismisses via `closeQuickOpen`, and awaits the promise — a one-liner replacing every `waitForHuman` Escape prompt
- Converted 33 `[assisted]` tests to automated across 9 files: `terminalPicker.test.ts` (12), `filePicker.test.ts` (8), `statusBarMenu.test.ts` (4), `coreSendCommands.test.ts` (3), `customAiAssistants.test.ts` (2), `bindToDestination.test.ts` (1), `editorBindingValidation.test.ts` (1), `clipboardPreservation.test.ts` (1), `goToRangeLink.test.ts` (1)
- Removed `StandardSuiteOptions` interface and `assisted` parameter from `standardSuite` — all 33 call sites are now two-argument form
- Removed `printAssistedBanner` function from `assistedTestHelper.ts` (dead code after option removal)
- Updated TESTING.md to document the `closeQuickOpen` dismissal pattern and `openAndDismiss` helper
- Flipped 33 QA YAML entries from `automated: assisted` to `automated: true`

## Test Plan

- [x] All 1874 unit tests pass (98.51% coverage)
- [x] QA validator passes (119 automated / 112 assisted / 4 false)
- [x] `test:release:automated` — 161 of 173 automated tests pass (12 pre-existing flaky failures in smartPadding, textEditorDestination, sendFilePath — unrelated to this change)

## Related

- Closes #557
…umber fixes, stale references

Added retry loop to openAndDismiss so the helper doesn't race against slow picker rendering on loaded CI. Extracted magic numbers into named constants per P003. Updated stale cross-references and test descriptions to reflect the switch from UI triggers to direct command invocation.

Ignored Feedback:
- Import order in editorBindingValidation.test.ts: False positive — assert is already at line 1 before all other imports.

Ref: #561 (review)
Converts the GitHub Copilot Chat destination picker visibility test from fully manual (automated: false) to fully automated using the openAndDismiss + extractQuickPickItemsLogged pattern. The test works without the Copilot Chat extension installed because isGitHubCopilotChatAvailable() detects the built-in workbench.action.chat.open command present in all modern VS Code versions.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts`:
- Around line 49-52: The helper can still hang because after breaking the loop
when Date.now() >= deadline you still await the original promise variable;
instead enforce a timeout when awaiting the command promise by racing it against
a timeout/rejection when the deadline passes (e.g. use Promise.race with a
timeout promise) so that the await after the loop cannot block indefinitely;
update the code that awaits promise and calls settle() to use this timeout-raced
promise and ensure any timeout path rejects or returns so settle() still runs
(references: the variables/function names deadline, promise, settle()).

In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/builtInAiAssistants.test.ts`:
- Line 5: The import line with multiple constants (CMD_BIND_TO_CLAUDE_CODE,
CMD_BIND_TO_DESTINATION, CMD_UNBIND_DESTINATION) fails Prettier; reformat the
import from '../../constants/commandIds' to the project style by wrapping the
named imports across multiple lines inside the braces (one per line or aligned
per project convention) and ensure proper trailing comma and spacing so Prettier
passes; update the import statement that references those constants
(CMD_BIND_TO_CLAUDE_CODE, CMD_BIND_TO_DESTINATION, CMD_UNBIND_DESTINATION)
accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 31443270-b63a-4b75-8033-c1fde108e584

📥 Commits

Reviewing files that changed from the base of the PR and between d2f22f3 and ce24f24.

📒 Files selected for processing (36)
  • CLAUDE.md
  • packages/rangelink-vscode-extension/TESTING.md
  • packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yaml
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/index.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/standardSuite.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/bindToDestination.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/builtInAiAssistants.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/commandRegistration.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorContent.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorTab.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuExplorer.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuTerminal.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/coreSendCommands.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/customAiAssistants.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/dirtyBufferWarning.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/editorBindingValidation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/filePathNavigation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/filePicker.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/filenameOnlyNavigation.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/goToRangeLink.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/linkGeneration.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationClamping.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationPrecision.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationToastSettings.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/platformKeybindings.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/releaseNotifier.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/sendFilePath.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/smartPadding.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/statusBarMenu.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/terminalPicker.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/textEditorDestination.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/unbind.test.ts
  • packages/rangelink-vscode-extension/src/__integration-tests__/suite/untitledNavigation.test.ts
✅ Files skipped from review due to trivial changes (8)
  • packages/rangelink-vscode-extension/src/integration-tests/suite/navigationPrecision.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/unbind.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/platformKeybindings.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/smartPadding.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/filenameOnlyNavigation.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/navigationClamping.test.ts
  • packages/rangelink-vscode-extension/TESTING.md
  • CLAUDE.md
🚧 Files skipped from review as they are similar to previous changes (21)
  • packages/rangelink-vscode-extension/src/integration-tests/suite/releaseNotifier.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/contextMenuEditorTab.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/commandRegistration.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/textEditorDestination.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/untitledNavigation.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/sendFilePath.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/linkGeneration.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/helpers/index.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/contextMenuExplorer.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/navigationToastSettings.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/editorBindingValidation.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/dirtyBufferWarning.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/helpers/assistedTestHelper.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/bindToDestination.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/coreSendCommands.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/goToRangeLink.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/clipboardPreservation.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/terminalPicker.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/statusBarMenu.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/customAiAssistants.test.ts
  • packages/rangelink-vscode-extension/src/integration-tests/suite/filePicker.test.ts

Comment thread packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts Outdated
…ently awaiting unresolved promise

When the retry loop hit the deadline, the old code broke out and then awaited the original promise, which was still pending. This hung until the Mocha timeout killed the entire test run. Now it throws immediately with a descriptive error so CI fails fast.

Benefits:
- CI fails in milliseconds instead of hanging until the Mocha timeout
- Error message includes the command name and deadline for fast diagnosis
- No silent failures — every timeout is a loud, actionable test failure

Ref: #561 (review)
@couimet couimet merged commit df17a83 into main May 13, 2026
3 checks passed
@couimet couimet deleted the issues/557 branch May 13, 2026 18:41
couimet added a commit that referenced this pull request May 14, 2026
…llback for AI assistant webview delivery

## Summary

The "single clipboard write per R-* operation" refactor (PR #556, `issues/547`) inadvertently reduced a three-command paste fallback chain to a single `editor.action.clipboardPasteAction`. That command only reaches VS Code text editors — for webview-hosted chat inputs (Claude Code, Cursor AI, GitHub Copilot Chat, custom AI Tier 1/2 webview targets) the system-level fallbacks `execPaste` / `paste` were what actually delivered content. Without them the dispatch returned success, but no content arrived in the chat input. This PR restores the fallback chain, logs the successful command for diagnostics, and converts the silent-pass `claude-code-004/005` tests to verdict-based assertions so the same class of regression cannot land green in the future.

## Changes

- Renamed `src/constants/pasteTimingConstants.ts` → `aiAssistantPasteConstants.ts` and added `AI_ASSISTANT_PASTE_COMMANDS = ['editor.action.clipboardPasteAction', 'execPaste', 'paste']` alongside the existing delay constants. The file now correctly reflects its scope (was a name-vs-content drift from #556 that removed the commands).
- `VscodeAdapter.pasteTextFromClipboard()` now iterates `AI_ASSISTANT_PASTE_COMMANDS` and stops on the first success. The `command` field is included in the `Clipboard paste succeeded` log so tests can assert which command delivered the paste. Per-command failures log at `debug` with `Paste command failed, trying next`; an all-failed path logs at `warn` with `allCommandsFailed: true`.
- Inline JSDoc on `pasteTextFromClipboard` explains WHY the fallback chain exists — non-obvious enough that the next refactor would otherwise repeat the same mistake.
- Added `assertPasteCommandLogged(lines, { command })` helper to `logBasedUiAssertions.ts` for integration-test assertions on which paste command succeeded.
- Updated `VscodeAdapter.test.ts` `pasteTextFromClipboard` describe block: 5 tests now cover the fallback chain (first-success, fall-through-to-second, all-fail, default post-paste delay, custom post-paste delay). All assertions include the `command` field per the test contract.
- Converted `claude-code-004` and `claude-code-005` from `waitForHuman` to `waitForHumanVerdict`. The silent-pass trap is closed for these specific TCs (broader audit tracked in #560). Setup is now programmatic (file open + selection via `vscode.workspace.openTextDocument` + `editor.selection`) and the send fires via `executeCommand(CMD_COPY_LINK_RELATIVE)` rather than asking the human to press the chord.
- Added the wave-1 Claude Code TCs from #483: `claude-code-001` (automated picker-inspection in `standardSuite`, uses `openAndDismiss` from #561), `claude-code-002` (assisted send-delivery with `waitForHumanVerdict`), and the deletion of `claude-code-003` (superseded by `claude-code-005`'s richer warm-path scenario). YAML flips for 001 → `automated: true` and 002 → `automated: assisted`.

## Key Discoveries

- The 547 refactor's "Test Plan" claimed `pnpm test:release passes` — technically true, but the with-extensions tests were not in that baseline AND `claude-code-004/005` used `waitForHuman` (Cancel-to-resolve) rather than `waitForHumanVerdict` (PASS/FAIL-to-resolve). Code-side log assertions stayed green even though the webview received nothing. The regression therefore landed undetected. The verdict-helper conversion in this PR closes that specific silent-pass trap; the broader audit lives in #560.
- The `dummy-ai-extension` fixture is already webview-based and exposes `dummyAi.getText` for reading what landed in the webview textareas — the missing piece for a fully automated webview paste-delivery test. Adding one is blocked on a new programmatic bind command for custom AI assistants (the current bind path requires a human picker selection); that command is out of scope here and will be filed as a follow-up issue.
- The 547 paste pipeline was audited beyond the bug-fix change: padding is correctly pre-applied at all 4 call sites (`LinkGenerator`, `TextSelectionPaster`, `FilePathPaster`, `TerminalSelectionService`); the two-delay model is used consistently in the single `pasteTextFromClipboard` chokepoint; the `focusCommands` fallback iteration in `AIAssistantFocusCapability` is intact. No additional regressions found.

## Test Plan

- [x] All existing unit tests pass — 107 suites, 228 tests in `VscodeAdapter.test.ts` (including 5 new fallback tests), 98.42% statement coverage.
- [x] `pnpm test:release:automated` — 165 of 175 passing. The 10 failures are pre-existing flakes (`langswitch-binding-*`, `smart-padding-*`, `stale-viewcolumn-001`) on non-AI-assistant code paths, same as noted in PR #561's description. My changes don't touch terminal/editor paste paths.
- [x] QA coverage validator passes — 121 automated / 115 assisted / 12 false markers all align with integration tests.
- [ ] **Assisted verification (definitive bug-fix proof):** `pnpm test:release:with-extensions --grep "claude-code-00[245]"` runs `claude-code-002` (cleanest reproduction), `claude-code-004` (cold panel paste), and `claude-code-005` (cold→warm sequence). Each prompts PASS/FAIL on whether the RangeLink actually arrived in Claude Code's chat input. All three must produce PASS verdicts.
- [ ] CI `test-with-extensions` job (from #564) runs green on the PR.

## Related

- Closes #559
- Resolves the regression introduced by #556
- Companion to #560 (silent-pass-trap audit) — this PR fixes the 547-introduced subset (`claude-code-004/005`); the broader audit remains open.
- Folds in the Wave-1 Claude Code work from #483 (`claude-code-001` / `claude-code-002` integration tests + the `claude-code-003` deletion).
- Complements #563 — this PR fixes the bug; #563 backfills coverage that would have caught it. They land independently; whichever merges first, the other rebases.
- Follow-up issue to be filed: a programmatic bind command for custom AI assistants (currently the bind path requires a human picker selection), which would unblock a fully automated webview paste-delivery test using the dummy-ai-extension's `dummyAi.getText` reader.
couimet added a commit that referenced this pull request May 14, 2026
…llback for AI assistant webview delivery (#566)

## Summary

The "single clipboard write per R-* operation" refactor (PR #556, `issues/547`) inadvertently reduced a three-command paste fallback chain to a single `editor.action.clipboardPasteAction`. That command only reaches VS Code text editors — for webview-hosted chat inputs (Claude Code, Cursor AI, GitHub Copilot Chat, custom AI Tier 1/2 webview targets) the system-level fallbacks `execPaste` / `paste` were what actually delivered content. Without them the dispatch returned success, but no content arrived in the chat input. This PR restores the fallback chain, logs the successful command for diagnostics, and converts the silent-pass `claude-code-004/005` tests to verdict-based assertions so the same class of regression cannot land green in the future.

## Changes

- Renamed `src/constants/pasteTimingConstants.ts` → `aiAssistantPasteConstants.ts` and added `AI_ASSISTANT_PASTE_COMMANDS = ['editor.action.clipboardPasteAction', 'execPaste', 'paste']` alongside the existing delay constants. The file now correctly reflects its scope (was a name-vs-content drift from #556 that removed the commands).
- `VscodeAdapter.pasteTextFromClipboard()` now iterates `AI_ASSISTANT_PASTE_COMMANDS` and stops on the first success. The `command` field is included in the `Clipboard paste succeeded` log so tests can assert which command delivered the paste. Per-command failures log at `debug` with `Paste command failed, trying next`; an all-failed path logs at `warn` with `allCommandsFailed: true`.
- Inline JSDoc on `pasteTextFromClipboard` explains WHY the fallback chain exists — non-obvious enough that the next refactor would otherwise repeat the same mistake.
- Added `assertPasteCommandLogged(lines, { command })` helper to `logBasedUiAssertions.ts` for integration-test assertions on which paste command succeeded.
- Updated `VscodeAdapter.test.ts` `pasteTextFromClipboard` describe block: 5 tests now cover the fallback chain (first-success, fall-through-to-second, all-fail, default post-paste delay, custom post-paste delay). All assertions include the `command` field per the test contract.
- Converted `claude-code-004` and `claude-code-005` from `waitForHuman` to `waitForHumanVerdict`. The silent-pass trap is closed for these specific TCs (broader audit tracked in #560). Setup is now programmatic (file open + selection via `vscode.workspace.openTextDocument` + `editor.selection`) and the send fires via `executeCommand(CMD_COPY_LINK_RELATIVE)` rather than asking the human to press the chord.
- Added the wave-1 Claude Code TCs from #483: `claude-code-001` (automated picker-inspection in `standardSuite`, uses `openAndDismiss` from #561), `claude-code-002` (assisted send-delivery with `waitForHumanVerdict`), and the deletion of `claude-code-003` (superseded by `claude-code-005`'s richer warm-path scenario). YAML flips for 001 → `automated: true` and 002 → `automated: assisted`.

## Key Discoveries

- The 547 refactor's "Test Plan" claimed `pnpm test:release passes` — technically true, but the with-extensions tests were not in that baseline AND `claude-code-004/005` used `waitForHuman` (Cancel-to-resolve) rather than `waitForHumanVerdict` (PASS/FAIL-to-resolve). Code-side log assertions stayed green even though the webview received nothing. The regression therefore landed undetected. The verdict-helper conversion in this PR closes that specific silent-pass trap; the broader audit lives in #560.
- The `dummy-ai-extension` fixture is already webview-based and exposes `dummyAi.getText` for reading what landed in the webview textareas — the missing piece for a fully automated webview paste-delivery test. Adding one is blocked on a new programmatic bind command for custom AI assistants (the current bind path requires a human picker selection); that command is out of scope here and will be filed as a follow-up issue.
- The 547 paste pipeline was audited beyond the bug-fix change: padding is correctly pre-applied at all 4 call sites (`LinkGenerator`, `TextSelectionPaster`, `FilePathPaster`, `TerminalSelectionService`); the two-delay model is used consistently in the single `pasteTextFromClipboard` chokepoint; the `focusCommands` fallback iteration in `AIAssistantFocusCapability` is intact. No additional regressions found.

## Test Plan

- [x] All existing unit tests pass — 107 suites, 228 tests in `VscodeAdapter.test.ts` (including 5 new fallback tests), 98.42% statement coverage.
- [x] `pnpm test:release:automated` — 165 of 175 passing. The 10 failures are pre-existing flakes (`langswitch-binding-*`, `smart-padding-*`, `stale-viewcolumn-001`) on non-AI-assistant code paths, same as noted in PR #561's description. My changes don't touch terminal/editor paste paths.
- [x] QA coverage validator passes — 121 automated / 115 assisted / 12 false markers all align with integration tests.
- [ ] **Assisted verification (definitive bug-fix proof):** `pnpm test:release:with-extensions --grep "claude-code-00[245]"` runs `claude-code-002` (cleanest reproduction), `claude-code-004` (cold panel paste), and `claude-code-005` (cold→warm sequence). Each prompts PASS/FAIL on whether the RangeLink actually arrived in Claude Code's chat input. All three must produce PASS verdicts.
- [ ] CI `test-with-extensions` job (from #564) runs green on the PR.

## Related

- Closes #559
- Resolves the regression introduced by #556
- Companion to #560 (silent-pass-trap audit) — this PR fixes the 547-introduced subset (`claude-code-004/005`); the broader audit remains open.
- Folds in the Wave-1 Claude Code work from #483 (`claude-code-001` / `claude-code-002` integration tests + the `claude-code-003` deletion).
- Complements #563 — this PR fixes the bug; #563 backfills coverage that would have caught it. They land independently; whichever merges first, the other rebases.
- Follow-up issue to be filed: a programmatic bind command for custom AI assistants (currently the bind path requires a human picker selection), which would unblock a fully automated webview paste-delivery test using the dummy-ai-extension's `dummyAi.getText` reader.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

closeQuickOpen automation survey — 41 assisted tests can become fully automated

1 participant