-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Problem
When an eval specifies multiple targets (e.g., vscode + copilot), the VSCode provider's sequential worker limit (workers=1) prevents other targets from running concurrently. The warning message is:
Warning: VSCode providers require window focus. Limiting workers from 3 to 1 to prevent race conditions.
This limit should only apply to VSCode provider instances, not to other providers like copilot that can run independently.
Expected Behavior
- VSCode evals should be serialized (1 worker for VSCode)
- Other targets (copilot, claude, etc.) should run concurrently alongside VSCode
- Each target type should have its own concurrency control
Actual Behavior
The worker limit is applied globally across all targets, so copilot waits for VSCode to finish (or vice versa).
Reproduction
execution:
targets:
- vscode
- copilot
tests:
- id: verify-multi-repo
input: Read package.json
assert:
- type: contains
value: agentvOnly VSCode runs; copilot appears to be blocked.
Context
Found during #458 — adding multi-target workspace example.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels