[issues/557] Automate 33 assisted integration tests via closeQuickOpen dismissal#561
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughAdds openAndDismiss(command), simplifies the standardSuite API (removes assisted/banner), documents programmatic QuickPick/InputBox dismissal, updates QA YAML entries, and converts many integration tests from assisted human steps to programmatic open→log→dismiss flows. ChangesTest Automation Refactor
Sequence DiagramsequenceDiagram
participant TestRunner
participant VSCode
participant Extension
TestRunner->>VSCode: executeCommand(CMD_...) // start command promise
TestRunner->>TestRunner: settle() // wait for logs to appear
loop while command pending
TestRunner->>VSCode: executeCommand('workbench.action.closeQuickOpen') // dismiss QuickPick/InputBox
end
VSCode->>Extension: command invoked (handler runs)
VSCode->>TestRunner: command promise resolves
TestRunner->>TestRunner: settle() // final wait for logs
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The PR updates the automation status of several test cases from 'assisted' to 'automated', indicating that they can now be fully automated. However, there may still be gaps in coverage for scenarios that require manual interaction. Suggested test cases:
Generated by QA Gap Check (GPT-4o-mini via GitHub Models) |
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (2)
packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts (1)
7-7: ⚡ Quick winRestore a named width constant for the section divider.
Inlining
60reintroduces a magic number and makes future width tuning brittle.♻️ Proposed fix
-const SECTION_LINE = '─'.repeat(60); +const SECTION_LINE_WIDTH = 60; +const SECTION_LINE = '─'.repeat(SECTION_LINE_WIDTH);As per coding guidelines, "Do not use magic numbers—define named constants for all numeric literals with semantic meaning using SCREAMING_SNAKE_CASE".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts` at line 7, Restore a named width constant for the section divider: introduce a SCREAMING_SNAKE_CASE constant (e.g., SECTION_WIDTH) and use it in the existing SECTION_LINE definition (SECTION_LINE = '─'.repeat(SECTION_WIDTH)) instead of inlining 60; update any related references or tests that assumed the numeric literal so future width tuning uses the single constant.packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts (1)
310-314: ⚡ Quick winExtract selection bounds into named constants.
Line 311–313 uses semantic numeric literals directly (
0,2). Please define local SCREAMING_SNAKE_CASE constants for these bounds to keep intent explicit and consistent.Proposed refactor
+ const SELECTION_START_LINE = 0; + const SELECTION_START_COLUMN = 0; + const SELECTION_END_LINE = 2; + const SELECTION_END_COLUMN = 0; const editor009 = await openEditor(fileUri); editor009.selection = new vscode.Selection( - new vscode.Position(0, 0), - new vscode.Position(2, 0), + new vscode.Position(SELECTION_START_LINE, SELECTION_START_COLUMN), + new vscode.Position(SELECTION_END_LINE, SELECTION_END_COLUMN), );As per coding guidelines,
**/*.{ts,tsx,js,jsx}: “Do not use magic numbers—define named constants for all numeric literals with semantic meaning using SCREAMING_SNAKE_CASE”.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts` around lines 310 - 314, The selection uses inline numeric literals when constructing vscode.Selection/vscode.Position (in the test where editor009.selection is set); extract these magic numbers into descriptive SCREAMING_SNAKE_CASE constants (e.g. START_LINE, START_COLUMN, END_LINE, END_COLUMN) at the top of the test or just above the selection setup and then use those constants when creating the two vscode.Position instances and the vscode.Selection so the intent of 0 and 2 is explicit and follows the project's no-magic-numbers guideline.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts`:
- Around line 37-41: The openAndDismiss helper can hang because a single await
settle() + one close action races against the command promise; change
openAndDismiss to repeatedly attempt dismissal until the command promise
resolves (or a short timeout) by: call vscode.commands.executeCommand(command)
into promise, then loop: await settle(), invoke
vscode.commands.executeCommand('workbench.action.closeQuickOpen') (or a no-op if
not open), check if promise settled (use Promise.race with a small timeout or
inspect resolution), and break when the command completes; reference
openAndDismiss, settle and the 'workbench.action.closeQuickOpen' command to
locate and update the implementation.
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/editorBindingValidation.test.ts`:
- Line 20: The import order is wrong: move the "import assert from
'node:assert'" statement to the group of Node built-in imports so it appears
before any project or third-party imports to satisfy the lint/import-order rule;
locate the existing "import assert from 'node:assert'" line in the test file and
place it with the other built-ins (top of the import block) so built-ins precede
project imports.
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/statusBarMenu.test.ts`:
- Around line 33-38: The test names "status-bar-menu-002" and the similar one at
lines 67-78 claim to verify clicking the status bar item / Cmd+R Cmd+M
keybinding but they only call CMD_OPEN_STATUS_BAR_MENU via openAndDismiss;
either update the tests to actually exercise the UI/keybinding paths or change
their titles/assertions to reflect they invoke the command directly. Locate the
tests by the identifiers 'status-bar-menu-002' (and the second case around lines
67-78), and either (a) replace the openAndDismiss(CMD_OPEN_STATUS_BAR_MENU) call
with a real UI interaction such as clicking the status bar item helper (e.g.,
clickStatusBarItem or the existing helper that targets the status bar) or
simulating the keybinding sequence (e.g., sendKeybinding for Cmd+R then Cmd+M),
or (b) rename the test descriptions to indicate they are invoking the command
directly and adjust assertions accordingly so coverage claims match actual
behavior.
In `@packages/rangelink-vscode-extension/TESTING.md`:
- Line 78: Update the broken reference in CLAUDE.md that still points to
"QuickPick limitation": change the text at the referenced location to read "See
TESTING.md § \"QuickPick and InputBox dismissal\" for what can and cannot be
automated" or update any direct anchor link to use
`#quickpick-and-inputbox-dismissal`; ensure the section title/name in TESTING.md
("QuickPick and InputBox dismissal") is used exactly so the cross-reference
resolves.
---
Nitpick comments:
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.ts`:
- Line 7: Restore a named width constant for the section divider: introduce a
SCREAMING_SNAKE_CASE constant (e.g., SECTION_WIDTH) and use it in the existing
SECTION_LINE definition (SECTION_LINE = '─'.repeat(SECTION_WIDTH)) instead of
inlining 60; update any related references or tests that assumed the numeric
literal so future width tuning uses the single constant.
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.ts`:
- Around line 310-314: The selection uses inline numeric literals when
constructing vscode.Selection/vscode.Position (in the test where
editor009.selection is set); extract these magic numbers into descriptive
SCREAMING_SNAKE_CASE constants (e.g. START_LINE, START_COLUMN, END_LINE,
END_COLUMN) at the top of the test or just above the selection setup and then
use those constants when creating the two vscode.Position instances and the
vscode.Selection so the intent of 0 and 2 is explicit and follows the project's
no-magic-numbers guideline.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: c398d930-b65f-4304-b210-846638e5bb7f
📒 Files selected for processing (34)
packages/rangelink-vscode-extension/TESTING.mdpackages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yamlpackages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.tspackages/rangelink-vscode-extension/src/__integration-tests__/helpers/index.tspackages/rangelink-vscode-extension/src/__integration-tests__/helpers/standardSuite.tspackages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/bindToDestination.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/commandRegistration.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorContent.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorTab.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuExplorer.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuTerminal.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/coreSendCommands.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/customAiAssistants.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/dirtyBufferWarning.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/editorBindingValidation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/filePathNavigation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/filePicker.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/filenameOnlyNavigation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/goToRangeLink.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/linkGeneration.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationClamping.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationPrecision.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationToastSettings.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/platformKeybindings.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/releaseNotifier.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/sendFilePath.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/smartPadding.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/statusBarMenu.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/terminalPicker.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/textEditorDestination.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/unbind.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/untitledNavigation.test.ts
…umber fixes, stale references Added retry loop to openAndDismiss so the helper doesn't race against slow picker rendering on loaded CI. Extracted magic numbers into named constants per P003. Updated stale cross-references and test descriptions to reflect the switch from UI triggers to direct command invocation. Ignored Feedback: - Import order in editorBindingValidation.test.ts: False positive — assert is already at line 1 before all other imports. Ref: #561 (review)
…en` dismissal
## Summary
`workbench.action.closeQuickOpen` can programmatically dismiss VS Code QuickPicks and InputBoxes from tests — the human was only needed to press Escape, not to select items. This converts 33 `[assisted]` tests to fully automated across 9 test files, reducing the assisted count from 147 to 112 and increasing automated from 86 to 119. Also removes the now-dead `{ assisted: true }` option from `standardSuite` since it only printed a cosmetic banner — test filtering is handled by the `[assisted]` prefix in test names.
## Changes
- Added `openAndDismiss(command)` helper that fires a command, settles, dismisses via `closeQuickOpen`, and awaits the promise — a one-liner replacing every `waitForHuman` Escape prompt
- Converted 33 `[assisted]` tests to automated across 9 files: `terminalPicker.test.ts` (12), `filePicker.test.ts` (8), `statusBarMenu.test.ts` (4), `coreSendCommands.test.ts` (3), `customAiAssistants.test.ts` (2), `bindToDestination.test.ts` (1), `editorBindingValidation.test.ts` (1), `clipboardPreservation.test.ts` (1), `goToRangeLink.test.ts` (1)
- Removed `StandardSuiteOptions` interface and `assisted` parameter from `standardSuite` — all 33 call sites are now two-argument form
- Removed `printAssistedBanner` function from `assistedTestHelper.ts` (dead code after option removal)
- Updated TESTING.md to document the `closeQuickOpen` dismissal pattern and `openAndDismiss` helper
- Flipped 33 QA YAML entries from `automated: assisted` to `automated: true`
## Test Plan
- [x] All 1874 unit tests pass (98.51% coverage)
- [x] QA validator passes (119 automated / 112 assisted / 4 false)
- [x] `test:release:automated` — 161 of 173 automated tests pass (12 pre-existing flaky failures in smartPadding, textEditorDestination, sendFilePath — unrelated to this change)
## Related
- Closes #557
…umber fixes, stale references Added retry loop to openAndDismiss so the helper doesn't race against slow picker rendering on loaded CI. Extracted magic numbers into named constants per P003. Updated stale cross-references and test descriptions to reflect the switch from UI triggers to direct command invocation. Ignored Feedback: - Import order in editorBindingValidation.test.ts: False positive — assert is already at line 1 before all other imports. Ref: #561 (review)
Converts the GitHub Copilot Chat destination picker visibility test from fully manual (automated: false) to fully automated using the openAndDismiss + extractQuickPickItemsLogged pattern. The test works without the Copilot Chat extension installed because isGitHubCopilotChatAvailable() detects the built-in workbench.action.chat.open command present in all modern VS Code versions.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.ts`:
- Around line 49-52: The helper can still hang because after breaking the loop
when Date.now() >= deadline you still await the original promise variable;
instead enforce a timeout when awaiting the command promise by racing it against
a timeout/rejection when the deadline passes (e.g. use Promise.race with a
timeout promise) so that the await after the loop cannot block indefinitely;
update the code that awaits promise and calls settle() to use this timeout-raced
promise and ensure any timeout path rejects or returns so settle() still runs
(references: the variables/function names deadline, promise, settle()).
In
`@packages/rangelink-vscode-extension/src/__integration-tests__/suite/builtInAiAssistants.test.ts`:
- Line 5: The import line with multiple constants (CMD_BIND_TO_CLAUDE_CODE,
CMD_BIND_TO_DESTINATION, CMD_UNBIND_DESTINATION) fails Prettier; reformat the
import from '../../constants/commandIds' to the project style by wrapping the
named imports across multiple lines inside the braces (one per line or aligned
per project convention) and ensure proper trailing comma and spacing so Prettier
passes; update the import statement that references those constants
(CMD_BIND_TO_CLAUDE_CODE, CMD_BIND_TO_DESTINATION, CMD_UNBIND_DESTINATION)
accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 31443270-b63a-4b75-8033-c1fde108e584
📒 Files selected for processing (36)
CLAUDE.mdpackages/rangelink-vscode-extension/TESTING.mdpackages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yamlpackages/rangelink-vscode-extension/src/__integration-tests__/helpers/assistedTestHelper.tspackages/rangelink-vscode-extension/src/__integration-tests__/helpers/index.tspackages/rangelink-vscode-extension/src/__integration-tests__/helpers/standardSuite.tspackages/rangelink-vscode-extension/src/__integration-tests__/helpers/testEnv.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/bindToDestination.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/builtInAiAssistants.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/clipboardPreservation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/commandRegistration.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorContent.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuEditorTab.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuExplorer.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/contextMenuTerminal.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/coreSendCommands.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/customAiAssistants.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/dirtyBufferWarning.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/editorBindingValidation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/filePathNavigation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/filePicker.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/filenameOnlyNavigation.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/goToRangeLink.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/linkGeneration.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationClamping.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationPrecision.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/navigationToastSettings.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/platformKeybindings.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/releaseNotifier.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/sendFilePath.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/smartPadding.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/statusBarMenu.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/terminalPicker.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/textEditorDestination.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/unbind.test.tspackages/rangelink-vscode-extension/src/__integration-tests__/suite/untitledNavigation.test.ts
✅ Files skipped from review due to trivial changes (8)
- packages/rangelink-vscode-extension/src/integration-tests/suite/navigationPrecision.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/unbind.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/platformKeybindings.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/smartPadding.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/filenameOnlyNavigation.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/navigationClamping.test.ts
- packages/rangelink-vscode-extension/TESTING.md
- CLAUDE.md
🚧 Files skipped from review as they are similar to previous changes (21)
- packages/rangelink-vscode-extension/src/integration-tests/suite/releaseNotifier.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/contextMenuEditorTab.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/commandRegistration.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/textEditorDestination.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/untitledNavigation.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/sendFilePath.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/linkGeneration.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/helpers/index.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/contextMenuExplorer.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/navigationToastSettings.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/editorBindingValidation.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/dirtyBufferWarning.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/helpers/assistedTestHelper.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/bindToDestination.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/coreSendCommands.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/goToRangeLink.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/clipboardPreservation.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/terminalPicker.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/statusBarMenu.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/customAiAssistants.test.ts
- packages/rangelink-vscode-extension/src/integration-tests/suite/filePicker.test.ts
…ently awaiting unresolved promise When the retry loop hit the deadline, the old code broke out and then awaited the original promise, which was still pending. This hung until the Mocha timeout killed the entire test run. Now it throws immediately with a descriptive error so CI fails fast. Benefits: - CI fails in milliseconds instead of hanging until the Mocha timeout - Error message includes the command name and deadline for fast diagnosis - No silent failures — every timeout is a loud, actionable test failure Ref: #561 (review)
…llback for AI assistant webview delivery ## Summary The "single clipboard write per R-* operation" refactor (PR #556, `issues/547`) inadvertently reduced a three-command paste fallback chain to a single `editor.action.clipboardPasteAction`. That command only reaches VS Code text editors — for webview-hosted chat inputs (Claude Code, Cursor AI, GitHub Copilot Chat, custom AI Tier 1/2 webview targets) the system-level fallbacks `execPaste` / `paste` were what actually delivered content. Without them the dispatch returned success, but no content arrived in the chat input. This PR restores the fallback chain, logs the successful command for diagnostics, and converts the silent-pass `claude-code-004/005` tests to verdict-based assertions so the same class of regression cannot land green in the future. ## Changes - Renamed `src/constants/pasteTimingConstants.ts` → `aiAssistantPasteConstants.ts` and added `AI_ASSISTANT_PASTE_COMMANDS = ['editor.action.clipboardPasteAction', 'execPaste', 'paste']` alongside the existing delay constants. The file now correctly reflects its scope (was a name-vs-content drift from #556 that removed the commands). - `VscodeAdapter.pasteTextFromClipboard()` now iterates `AI_ASSISTANT_PASTE_COMMANDS` and stops on the first success. The `command` field is included in the `Clipboard paste succeeded` log so tests can assert which command delivered the paste. Per-command failures log at `debug` with `Paste command failed, trying next`; an all-failed path logs at `warn` with `allCommandsFailed: true`. - Inline JSDoc on `pasteTextFromClipboard` explains WHY the fallback chain exists — non-obvious enough that the next refactor would otherwise repeat the same mistake. - Added `assertPasteCommandLogged(lines, { command })` helper to `logBasedUiAssertions.ts` for integration-test assertions on which paste command succeeded. - Updated `VscodeAdapter.test.ts` `pasteTextFromClipboard` describe block: 5 tests now cover the fallback chain (first-success, fall-through-to-second, all-fail, default post-paste delay, custom post-paste delay). All assertions include the `command` field per the test contract. - Converted `claude-code-004` and `claude-code-005` from `waitForHuman` to `waitForHumanVerdict`. The silent-pass trap is closed for these specific TCs (broader audit tracked in #560). Setup is now programmatic (file open + selection via `vscode.workspace.openTextDocument` + `editor.selection`) and the send fires via `executeCommand(CMD_COPY_LINK_RELATIVE)` rather than asking the human to press the chord. - Added the wave-1 Claude Code TCs from #483: `claude-code-001` (automated picker-inspection in `standardSuite`, uses `openAndDismiss` from #561), `claude-code-002` (assisted send-delivery with `waitForHumanVerdict`), and the deletion of `claude-code-003` (superseded by `claude-code-005`'s richer warm-path scenario). YAML flips for 001 → `automated: true` and 002 → `automated: assisted`. ## Key Discoveries - The 547 refactor's "Test Plan" claimed `pnpm test:release passes` — technically true, but the with-extensions tests were not in that baseline AND `claude-code-004/005` used `waitForHuman` (Cancel-to-resolve) rather than `waitForHumanVerdict` (PASS/FAIL-to-resolve). Code-side log assertions stayed green even though the webview received nothing. The regression therefore landed undetected. The verdict-helper conversion in this PR closes that specific silent-pass trap; the broader audit lives in #560. - The `dummy-ai-extension` fixture is already webview-based and exposes `dummyAi.getText` for reading what landed in the webview textareas — the missing piece for a fully automated webview paste-delivery test. Adding one is blocked on a new programmatic bind command for custom AI assistants (the current bind path requires a human picker selection); that command is out of scope here and will be filed as a follow-up issue. - The 547 paste pipeline was audited beyond the bug-fix change: padding is correctly pre-applied at all 4 call sites (`LinkGenerator`, `TextSelectionPaster`, `FilePathPaster`, `TerminalSelectionService`); the two-delay model is used consistently in the single `pasteTextFromClipboard` chokepoint; the `focusCommands` fallback iteration in `AIAssistantFocusCapability` is intact. No additional regressions found. ## Test Plan - [x] All existing unit tests pass — 107 suites, 228 tests in `VscodeAdapter.test.ts` (including 5 new fallback tests), 98.42% statement coverage. - [x] `pnpm test:release:automated` — 165 of 175 passing. The 10 failures are pre-existing flakes (`langswitch-binding-*`, `smart-padding-*`, `stale-viewcolumn-001`) on non-AI-assistant code paths, same as noted in PR #561's description. My changes don't touch terminal/editor paste paths. - [x] QA coverage validator passes — 121 automated / 115 assisted / 12 false markers all align with integration tests. - [ ] **Assisted verification (definitive bug-fix proof):** `pnpm test:release:with-extensions --grep "claude-code-00[245]"` runs `claude-code-002` (cleanest reproduction), `claude-code-004` (cold panel paste), and `claude-code-005` (cold→warm sequence). Each prompts PASS/FAIL on whether the RangeLink actually arrived in Claude Code's chat input. All three must produce PASS verdicts. - [ ] CI `test-with-extensions` job (from #564) runs green on the PR. ## Related - Closes #559 - Resolves the regression introduced by #556 - Companion to #560 (silent-pass-trap audit) — this PR fixes the 547-introduced subset (`claude-code-004/005`); the broader audit remains open. - Folds in the Wave-1 Claude Code work from #483 (`claude-code-001` / `claude-code-002` integration tests + the `claude-code-003` deletion). - Complements #563 — this PR fixes the bug; #563 backfills coverage that would have caught it. They land independently; whichever merges first, the other rebases. - Follow-up issue to be filed: a programmatic bind command for custom AI assistants (currently the bind path requires a human picker selection), which would unblock a fully automated webview paste-delivery test using the dummy-ai-extension's `dummyAi.getText` reader.
…llback for AI assistant webview delivery (#566) ## Summary The "single clipboard write per R-* operation" refactor (PR #556, `issues/547`) inadvertently reduced a three-command paste fallback chain to a single `editor.action.clipboardPasteAction`. That command only reaches VS Code text editors — for webview-hosted chat inputs (Claude Code, Cursor AI, GitHub Copilot Chat, custom AI Tier 1/2 webview targets) the system-level fallbacks `execPaste` / `paste` were what actually delivered content. Without them the dispatch returned success, but no content arrived in the chat input. This PR restores the fallback chain, logs the successful command for diagnostics, and converts the silent-pass `claude-code-004/005` tests to verdict-based assertions so the same class of regression cannot land green in the future. ## Changes - Renamed `src/constants/pasteTimingConstants.ts` → `aiAssistantPasteConstants.ts` and added `AI_ASSISTANT_PASTE_COMMANDS = ['editor.action.clipboardPasteAction', 'execPaste', 'paste']` alongside the existing delay constants. The file now correctly reflects its scope (was a name-vs-content drift from #556 that removed the commands). - `VscodeAdapter.pasteTextFromClipboard()` now iterates `AI_ASSISTANT_PASTE_COMMANDS` and stops on the first success. The `command` field is included in the `Clipboard paste succeeded` log so tests can assert which command delivered the paste. Per-command failures log at `debug` with `Paste command failed, trying next`; an all-failed path logs at `warn` with `allCommandsFailed: true`. - Inline JSDoc on `pasteTextFromClipboard` explains WHY the fallback chain exists — non-obvious enough that the next refactor would otherwise repeat the same mistake. - Added `assertPasteCommandLogged(lines, { command })` helper to `logBasedUiAssertions.ts` for integration-test assertions on which paste command succeeded. - Updated `VscodeAdapter.test.ts` `pasteTextFromClipboard` describe block: 5 tests now cover the fallback chain (first-success, fall-through-to-second, all-fail, default post-paste delay, custom post-paste delay). All assertions include the `command` field per the test contract. - Converted `claude-code-004` and `claude-code-005` from `waitForHuman` to `waitForHumanVerdict`. The silent-pass trap is closed for these specific TCs (broader audit tracked in #560). Setup is now programmatic (file open + selection via `vscode.workspace.openTextDocument` + `editor.selection`) and the send fires via `executeCommand(CMD_COPY_LINK_RELATIVE)` rather than asking the human to press the chord. - Added the wave-1 Claude Code TCs from #483: `claude-code-001` (automated picker-inspection in `standardSuite`, uses `openAndDismiss` from #561), `claude-code-002` (assisted send-delivery with `waitForHumanVerdict`), and the deletion of `claude-code-003` (superseded by `claude-code-005`'s richer warm-path scenario). YAML flips for 001 → `automated: true` and 002 → `automated: assisted`. ## Key Discoveries - The 547 refactor's "Test Plan" claimed `pnpm test:release passes` — technically true, but the with-extensions tests were not in that baseline AND `claude-code-004/005` used `waitForHuman` (Cancel-to-resolve) rather than `waitForHumanVerdict` (PASS/FAIL-to-resolve). Code-side log assertions stayed green even though the webview received nothing. The regression therefore landed undetected. The verdict-helper conversion in this PR closes that specific silent-pass trap; the broader audit lives in #560. - The `dummy-ai-extension` fixture is already webview-based and exposes `dummyAi.getText` for reading what landed in the webview textareas — the missing piece for a fully automated webview paste-delivery test. Adding one is blocked on a new programmatic bind command for custom AI assistants (the current bind path requires a human picker selection); that command is out of scope here and will be filed as a follow-up issue. - The 547 paste pipeline was audited beyond the bug-fix change: padding is correctly pre-applied at all 4 call sites (`LinkGenerator`, `TextSelectionPaster`, `FilePathPaster`, `TerminalSelectionService`); the two-delay model is used consistently in the single `pasteTextFromClipboard` chokepoint; the `focusCommands` fallback iteration in `AIAssistantFocusCapability` is intact. No additional regressions found. ## Test Plan - [x] All existing unit tests pass — 107 suites, 228 tests in `VscodeAdapter.test.ts` (including 5 new fallback tests), 98.42% statement coverage. - [x] `pnpm test:release:automated` — 165 of 175 passing. The 10 failures are pre-existing flakes (`langswitch-binding-*`, `smart-padding-*`, `stale-viewcolumn-001`) on non-AI-assistant code paths, same as noted in PR #561's description. My changes don't touch terminal/editor paste paths. - [x] QA coverage validator passes — 121 automated / 115 assisted / 12 false markers all align with integration tests. - [ ] **Assisted verification (definitive bug-fix proof):** `pnpm test:release:with-extensions --grep "claude-code-00[245]"` runs `claude-code-002` (cleanest reproduction), `claude-code-004` (cold panel paste), and `claude-code-005` (cold→warm sequence). Each prompts PASS/FAIL on whether the RangeLink actually arrived in Claude Code's chat input. All three must produce PASS verdicts. - [ ] CI `test-with-extensions` job (from #564) runs green on the PR. ## Related - Closes #559 - Resolves the regression introduced by #556 - Companion to #560 (silent-pass-trap audit) — this PR fixes the 547-introduced subset (`claude-code-004/005`); the broader audit remains open. - Folds in the Wave-1 Claude Code work from #483 (`claude-code-001` / `claude-code-002` integration tests + the `claude-code-003` deletion). - Complements #563 — this PR fixes the bug; #563 backfills coverage that would have caught it. They land independently; whichever merges first, the other rebases. - Follow-up issue to be filed: a programmatic bind command for custom AI assistants (currently the bind path requires a human picker selection), which would unblock a fully automated webview paste-delivery test using the dummy-ai-extension's `dummyAi.getText` reader.
Summary
workbench.action.closeQuickOpencan programmatically dismiss VS Code QuickPicks and InputBoxes from tests — the human was only needed to press Escape, not to select items. This converts 33[assisted]tests to fully automated across 9 test files, reducing the assisted count from 147 to 112 and increasing automated from 86 to 119. Also removes the now-dead{ assisted: true }option fromstandardSuitesince it only printed a cosmetic banner — test filtering is handled by the[assisted]prefix in test names.Changes
openAndDismiss(command)helper that fires a command, settles, dismisses viacloseQuickOpen, and awaits the promise — a one-liner replacing everywaitForHumanEscape prompt[assisted]tests to automated across 9 files:terminalPicker.test.ts(12),filePicker.test.ts(8),statusBarMenu.test.ts(4),coreSendCommands.test.ts(3),customAiAssistants.test.ts(2),bindToDestination.test.ts(1),editorBindingValidation.test.ts(1),clipboardPreservation.test.ts(1),goToRangeLink.test.ts(1)StandardSuiteOptionsinterface andassistedparameter fromstandardSuite— all 33 call sites are now two-argument formprintAssistedBannerfunction fromassistedTestHelper.ts(dead code after option removal)closeQuickOpendismissal pattern andopenAndDismisshelperautomated: assistedtoautomated: trueTest Plan
test:release:automated— 161 of 173 automated tests pass (12 pre-existing flaky failures in smartPadding, textEditorDestination, sendFilePath — unrelated to this change)Related
closeQuickOpenautomation survey — 41 assisted tests can become fully automated #557Summary by CodeRabbit
Tests
Documentation