feat(sdk): make the chat.agent system prompt cacheable#3952
Merged
Conversation
chat.toStreamTextOptions() can now emit the system prompt as a structured message carrying providerOptions, so a provider can cache the system block. Opt in three ways (most specific wins): cacheControl sugar or systemProviderOptions on toStreamTextOptions(), or providerOptions on chat.prompt.set(). Without an option, system stays a plain string.
🦋 Changeset detectedLatest commit: 981d9bd The changes in this PR will be included in the next version bump. This PR includes changesets to release 25 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📜 Recent review details⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (33)
WalkthroughThis patch introduces opt-in Anthropic prompt caching for the chat agent's system prompt in 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install timed out. The project may have too many dependencies for the sandbox. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…roviderOptions ai re-exports ProviderMetadata (same Record<string, Record<string, JSONValue>> shape) but not ProviderOptions, so importing the latter failed the SDK build with TS2459. The public option names are unchanged.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
A later chat.prompt.set() with no options left the previous prompt's providerOptions in locals, so toStreamTextOptions() could still cache a prompt that did not opt in. Always overwrite the slot. Also replaces fixed sleeps in the prompt-caching test with a bounded condition wait.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
nicktrn
approved these changes
Jun 15, 2026
ericallam
added a commit
that referenced
this pull request
Jun 15, 2026
…locks (#3954) ## Summary The script that generates the changeset release PR description was silently dropping some changelog entries and stripping code examples. In [#3932](#3932), entry [#3937](#3937) was missing entirely from the Improvements list and [#3952](#3952 code block was gone, even though both were present in the raw changeset output. ## Root cause `parsePrBody` parsed the raw changeset body line by line: - The dependency-bump filter matched any entry whose text *began* with a backticked package name, so a real changelog entry like `` `@trigger.dev/sdk` now bundles... `` got thrown out along with the genuine version-bump lines. - Only the first line of each bullet was kept, so fenced code blocks, sub-bullets, and continuation paragraphs were discarded. ## Fix Group each top-level bullet with its indented continuation (code blocks, sub-bullets, paragraphs), dedent it, and re-emit it intact. The dependency filter is now anchored so it only matches lines that are *entirely* a package bump, leaving real entries that merely start with a package name. Verified by replaying #3932's raw body through the script: #3937 returns to the list, #3952's code block is preserved, and #3936's sub-bullets nest correctly under their parent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
chat.agent's system prompt (thechat.prompttext plus any skills preamble) could not carry a provider cache breakpoint, so the largest and most stable part of the prompt re-paid full input price on every turn.chat.toStreamTextOptions()now emits the system prompt as a structured message carryingproviderOptionswhen you opt in, so a provider can cache the system block. Without an option,systemstays a plain string, so existing behavior is unchanged.API
Three ways to opt in (most specific wins, no deep merge):
The
cacheControlshorthand is Anthropic-only;systemProviderOptionsis the general form. Pairs with aprepareMessagescache breakpoint to cache the conversation prefix too.Docs guide: #3951