[issues/562] Add CI job for integration tests with marketplace extensions#564
Open
couimet wants to merge 2 commits into
Open
[issues/562] Add CI job for integration tests with marketplace extensions#564couimet wants to merge 2 commits into
couimet wants to merge 2 commits into
Conversation
…ions ## Summary Adds a dedicated CI leg that runs `pnpm test:release:with-extensions --no-assisted` under Xvfb. Before this, that test mode only ran locally — the custom AI assistant tests depend on a `dummy-ai-extension` installed from marketplace, so the standard `test:release:automated` CI leg (which skips extension install) could not cover them. ## Changes - Added `.github/actions/run-integration-tests-with-extensions/action.yml` — composite action that installs Xvfb + native deps, caches the VS Code test binary, and runs the extension-dependent integration tests - Appended `test-with-extensions` job to `.github/workflows/ci.yml` — runs on `ubuntu-latest` in parallel with the existing `test` job ## Test Plan - [ ] CI: push branch, confirm the `test-with-extensions` job runs green on GitHub Actions ## Related - Closes #562
Contributor
WalkthroughA new composite GitHub Action and CI job run the VS Code extension-dependent integration tests under Xvfb with native deps installed and ChangesIntegration Tests with Extensions CI
VS Code test config refactor and runner script
🎯 3 (Moderate) | ⏱️ ~20 minutes
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/actions/run-integration-tests-with-extensions/action.yml:
- Line 10: The apt install line hardcodes libasound2t64 which is missing on
ubuntu-22.04; change the action step so it first probes package availability
(e.g., using apt-cache show or apt-cache policy) and picks libasound2t64 if
present otherwise libasound2, then run sudo apt-get install -y
--no-install-recommends xvfb libgtk-3-0 libxss1 libnss3 plus the chosen ALSA
package; update the install step that currently contains the apt-get command to
use this conditional selection so the job works on both ubuntu-22.04 and
ubuntu-24.04.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: ce06a9eb-dd8a-4545-8495-4847d529d5c5
📒 Files selected for processing (2)
.github/actions/run-integration-tests-with-extensions/action.yml.github/workflows/ci.yml
…ep to shell script Each vscode-test config duplicated the same ~20 lines of boilerplate (files, extensionDevelopmentPath, workspaceFolder, version, launchArgs, env, mocha). Before this change, the automated config also hardcoded the grep/invert for [assisted] while with-extensions and the default config duplicated the env-based grep logic. Extracting a base config and delegating filtering to the shell script makes the config files dumb pipes — they spread BASE_CONFIG, override one or two fields, and done. Benefits: - Config files shrink from ~28 lines to ~10 lines each - Single source of truth for mocha grep/invert — MOCHA_GREP/MOCHA_INVERT env vars set by test-release-run.sh - Adding a new vscode-test config is one import + one spread, not copy-paste - [assisted] pattern lives only in the shell script that defines --automated and --no-assisted switches
Owner
Author
|
@coderabbitai review -- I see you skipped in |
Contributor
|
✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/rangelink-vscode-extension/scripts/test-release-run.sh`:
- Around line 108-111: The script currently sets MOCHA_INVERT=true whenever
MODE=automated (or NO_ASSISTED true), which later causes a user-supplied
GREP_PATTERN to be inverted; change the condition that sets MOCHA_GREP and
MOCHA_INVERT so it only runs when no user grep/label is provided. Specifically,
update the block that exports MOCHA_GREP and MOCHA_INVERT (the one referencing
MODE, NO_ASSISTED, MOCHA_GREP and MOCHA_INVERT) to check that GREP_PATTERN is
empty before setting the assisted pattern and inversion (e.g. only set when -z
"$GREP_PATTERN"), and make the same adjustment for the duplicate logic around
the later lines that also set MOCHA_GREP/MOCHA_INVERT.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 5e6bcc5c-bd15-4f8b-a18b-97e898855468
📒 Files selected for processing (8)
.github/actions/run-integration-tests-with-extensions/action.yml.github/workflows/ci.ymlpackages/rangelink-vscode-extension/.vscode-test.automated.mjspackages/rangelink-vscode-extension/.vscode-test.base.mjspackages/rangelink-vscode-extension/.vscode-test.mjspackages/rangelink-vscode-extension/.vscode-test.with-extensions.mjspackages/rangelink-vscode-extension/TESTING.mdpackages/rangelink-vscode-extension/scripts/test-release-run.sh
Comment on lines
+108
to
+111
| if [[ "$MODE" == "automated" ]] || [[ "$NO_ASSISTED" == true && -z "$GREP_PATTERN" ]]; then | ||
| export MOCHA_GREP='\[assisted\]' | ||
| export MOCHA_INVERT=true | ||
| fi |
Contributor
There was a problem hiding this comment.
Automated + grep currently inverts the user’s filter unexpectedly.
When MODE=automated, Line 108 sets MOCHA_INVERT=true and Line 109 sets the assisted pattern, but Line 150 later replaces MOCHA_GREP with the user pattern. That leaves inversion enabled against the user pattern (run “not grep”), which is incorrect for --automated --grep ... and --automated --label ... flows.
Proposed fix
-if [[ "$MODE" == "automated" ]] || [[ "$NO_ASSISTED" == true && -z "$GREP_PATTERN" ]]; then
+if [[ -z "$GREP_PATTERN" ]] && ([[ "$MODE" == "automated" ]] || [[ "$NO_ASSISTED" == true ]]); then
export MOCHA_GREP='\[assisted\]'
export MOCHA_INVERT=true
fiAlso applies to: 149-151
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/rangelink-vscode-extension/scripts/test-release-run.sh` around lines
108 - 111, The script currently sets MOCHA_INVERT=true whenever MODE=automated
(or NO_ASSISTED true), which later causes a user-supplied GREP_PATTERN to be
inverted; change the condition that sets MOCHA_GREP and MOCHA_INVERT so it only
runs when no user grep/label is provided. Specifically, update the block that
exports MOCHA_GREP and MOCHA_INVERT (the one referencing MODE, NO_ASSISTED,
MOCHA_GREP and MOCHA_INVERT) to check that GREP_PATTERN is empty before setting
the assisted pattern and inversion (e.g. only set when -z "$GREP_PATTERN"), and
make the same adjustment for the duplicate logic around the later lines that
also set MOCHA_GREP/MOCHA_INVERT.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a dedicated CI leg that runs
pnpm test:release:with-extensions --no-assistedunder Xvfb. Before this, that test mode only ran locally — the custom AI assistant tests depend on adummy-ai-extensioninstalled from marketplace, so the standardtest:release:automatedCI leg (which skips extension install) could not cover them.Changes
.github/actions/run-integration-tests-with-extensions/action.yml— composite action that installs Xvfb + native deps, caches the VS Code test binary, and runs the extension-dependent integration teststest-with-extensionsjob to.github/workflows/ci.yml— runs onubuntu-latestin parallel with the existingtestjobTest Plan
test-with-extensionsjob runs green on GitHub ActionsRelated
Summary by CodeRabbit
Chores
Tests
Documentation