fix(core): place cache_control at content block level per DashScope API spec#1377
Open
flystar32 wants to merge 1 commit into
Open
fix(core): place cache_control at content block level per DashScope API spec#1377flystar32 wants to merge 1 commit into
flystar32 wants to merge 1 commit into
Conversation
…PI spec (agentscope-ai#1363) The DashScope API documentation requires `cache_control` to be placed inside content blocks (within the `content` array), not at the message level. The previous implementation incorrectly set `cache_control` as a sibling of `role` and `content`, which prevented context caching from working properly. Changes: - Add `cacheControl` field to DashScopeContentPart and OpenAIContentPart DTOs - Rewrite applyCacheControl() in DashScopeChatFormatter, DashScopeMultiAgentFormatter, and OpenAIBaseFormatter to set cache_control on the last content block of target messages, converting string content to array format when needed - Update applyCacheControlFromMetadata() in DashScopeMessageConverter and OpenAIMessageConverter to use content block level placement - Rewrite DashScopeCacheControlTest and OpenAICacheControlTest with 29 test cases validating content block level placement, string-to-array conversion, and manual marking preservation - Add CacheControlE2ETest verifying both DashScope native and OpenAI-compatible endpoints accept the corrected format Closes agentscope-ai#1363 Change-Id: Ibf3c1a9bf669fb03c3759315ca787edc9eacf162 Co-developed-by: Cursor <noreply@cursor.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes DashScope context caching by moving cache_control from the message level into the last content block (inside the content array), matching DashScope’s API requirements and applying the same behavior to the OpenAI-compatible request path.
Changes:
- Add
cache_controlsupport to DashScope/OpenAI content-part DTOs. - Update DashScope/OpenAI formatters and message converters to set
cache_controlon the last content block and convert stringcontentto array form when needed. - Add/expand unit tests and introduce an E2E test covering both DashScope native and compatible-mode endpoints.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| agentscope-core/src/main/java/io/agentscope/core/formatter/dashscope/DashScopeChatFormatter.java | Applies cache control at content-block level and converts string content to array form. |
| agentscope-core/src/main/java/io/agentscope/core/formatter/dashscope/DashScopeMultiAgentFormatter.java | Delegates cache-control application to DashScopeChatFormatter block-level helper. |
| agentscope-core/src/main/java/io/agentscope/core/formatter/dashscope/DashScopeMessageConverter.java | Moves metadata-driven cache control to content-block level. |
| agentscope-core/src/main/java/io/agentscope/core/formatter/dashscope/dto/DashScopeContentPart.java | Adds cacheControl field on content parts for correct serialization. |
| agentscope-core/src/main/java/io/agentscope/core/formatter/openai/OpenAIBaseFormatter.java | Applies cache control at content-block level and converts string content to array form. |
| agentscope-core/src/main/java/io/agentscope/core/formatter/openai/OpenAIMessageConverter.java | Moves metadata-driven cache control to content-block level. |
| agentscope-core/src/main/java/io/agentscope/core/formatter/openai/dto/OpenAIContentPart.java | Adds cacheControl field on content parts for correct serialization. |
| agentscope-core/src/test/java/io/agentscope/core/formatter/dashscope/DashScopeCacheControlTest.java | Updates/extends tests for block-level placement and serialization expectations. |
| agentscope-core/src/test/java/io/agentscope/core/formatter/openai/OpenAICacheControlTest.java | Updates/extends tests for block-level placement and string→array conversion. |
| agentscope-core/src/test/java/io/agentscope/core/e2e/CacheControlE2ETest.java | Adds E2E coverage for DashScope native + compatible-mode endpoints. |
Comment on lines
+203
to
+215
| * Apply ephemeral cache_control to the last content block of the given message. | ||
| * If content is a plain string, it is first converted to array format. | ||
| * Skips if the last content block already has cache_control set. | ||
| */ | ||
| static void applyCacheControlToContentBlock(DashScopeMessage msg) { | ||
| List<DashScopeContentPart> parts = ensureContentArray(msg); | ||
| if (parts.isEmpty()) { | ||
| return; | ||
| } | ||
| DashScopeContentPart lastPart = parts.get(parts.size() - 1); | ||
| if (lastPart.getCacheControl() == null) { | ||
| lastPart.setCacheControl(EPHEMERAL_CACHE_CONTROL); | ||
| } |
Comment on lines
+201
to
+212
| * Apply ephemeral cache_control to the last content block of the given message. | ||
| * If content is a plain string, it is first converted to array format. | ||
| * Skips if the last content block already has cache_control set. | ||
| */ | ||
| static void applyCacheControlToContentBlock(OpenAIMessage msg) { | ||
| List<OpenAIContentPart> parts = ensureContentArray(msg); | ||
| if (parts.isEmpty()) { | ||
| return; | ||
| } | ||
| OpenAIContentPart lastPart = parts.get(parts.size() - 1); | ||
| if (lastPart.getCacheControl() == null) { | ||
| lastPart.setCacheControl(EPHEMERAL_CACHE_CONTROL); |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1363
Per the DashScope API documentation,
cache_controlmust be placed inside content blocks (within thecontentarray), not at the message level. The previous implementation incorrectly setcache_controlas a sibling ofroleandcontent, which prevented context caching from working properly.This fix applies to both DashScope native and OpenAI-compatible protocol paths.
Changes
DTO layer (2 files):
DashScopeContentPart— addcacheControlfield with getter/setter/builderOpenAIContentPart— same as aboveFormatter layer (4 files):
DashScopeChatFormatter.applyCacheControl()— rewrite to setcache_controlon the last content block of target messages; addapplyCacheControlToContentBlock()andensureContentArray()utility methods (auto-converts string content to array format)DashScopeMultiAgentFormatter.applyCacheControl()— delegate toDashScopeChatFormatter's static methodsOpenAIBaseFormatter.applyCacheControl()— symmetric changes for OpenAI pathDashScopeMessageConverter/OpenAIMessageConverter— updateapplyCacheControlFromMetadata()to use content block level placementBefore (incorrect — message level):
{ "role": "system", "content": "You are helpful.", "cache_control": {"type": "ephemeral"} }After (correct — content block level):
{ "role": "system", "content": [ {"text": "You are helpful.", "cache_control": {"type": "ephemeral"}} ] }Test plan
DashScopeCacheControlTest— 16 test cases covering content block level placement, string→array conversion, manual marking preservation, JSON serialization, metadata markingOpenAICacheControlTest— 13 test cases covering the same scenarios for OpenAI pathCacheControlE2ETest— real API calls to both DashScope native and OpenAI-compatible endpoints (requiresDASHSCOPE_API_KEY+ENABLE_E2E_TESTS=true)DashScopeChatModelTest,OpenAIChatModelTest) and ground truth tests pass/compatible-mode/v1) accept the corrected format and return successful responsesMade with Cursor