Skip to content

feat: add maxConcurrency option for providers#27938

Open
tjokinen wants to merge 1 commit into
anomalyco:devfrom
tjokinen:fix/provider-max-concurrency
Open

feat: add maxConcurrency option for providers#27938
tjokinen wants to merge 1 commit into
anomalyco:devfrom
tjokinen:fix/provider-max-concurrency

Conversation

@tjokinen
Copy link
Copy Markdown

Issue for this PR

Closes #12019

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Adds an optional maxConcurrency field to provider options. When set, opencode holds a per-provider semaphore permit for the whole lifetime of each LLM.stream call, so providers with a concurrent-request cap stop rejecting overlapping requests. In my case: Featherless + Kimi K2.5, where the title-generation stream opencode fires alongside the main response is enough on its own to exceed the account's 4-unit limit.

The permit is held via Effect.acquireRelease inside Stream.scoped, not by wrapping streamText. The HTTP request outlives the synchronous call, so the permit has to outlive it too.

Example (opencode.json):

  {
    "provider": {
      "featherless": {
        "options": {
          "maxConcurrency": 1
        }
      }
    }
  }

Omitting the field keeps current behavior (no limit).

How did you verify your code works?

  • bun typecheck clean.
  • 4 unit tests in test/provider/concurrency.test.ts covering serialization, scope-bound release, per-provider isolation, undefined passthrough.
  • Configured maxConcurrency: 1 for Featherless locally and confirmed the "Concurrency limit exceeded" error no longer fires.

Screenshots / recordings

N/A - no UI changes.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions
Copy link
Copy Markdown
Contributor

Hey! Your PR title add maxConcurrency option for providers doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

@tjokinen tjokinen changed the title add maxConcurrency option for providers feat: add maxConcurrency option for providers May 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Configure model concurrency

1 participant