fix(bedrock): drop metrics-only invocation trailer from stream decoder#1702
Open
camposvinicius wants to merge 1 commit into
Open
fix(bedrock): drop metrics-only invocation trailer from stream decoder#1702camposvinicius wants to merge 1 commit into
camposvinicius wants to merge 1 commit into
Conversation
Follow-up to anthropics#1682, which preserved the real event type but still forwarded the amazon-bedrock-invocationMetrics trailer as event="completion". That trailer carries no type and no completion field, so on a Messages stream it constructs against the RawMessageStreamEvent union with no discriminator, falls back to the first union member, and yields a contract-violating RawMessageStartEvent(message=None). Consumers that trust the type annotations (for example event.message.usage) then crash with AttributeError. The decoder now drops the metrics-only trailer instead of forwarding it. Typed Messages events, legacy text completions, and legacy completions that also carry the metrics trailer are all unchanged. Adds regression tests covering the drop and documenting the union fallback the drop protects against. Fixes anthropics#1647.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Follow-up to #1682. That PR fixed typed-event routing by preserving each chunk's real event type, but the
amazon-bedrock-invocationMetricstrailer is still forwarded asevent="completion". The trailer carries neither atypenor acompletionfield, so on a Messages stream the"completion"branch in_streaming.py::__stream__constructs it against theRawMessageStreamEventunion with no discriminator. The union falls back to its first member and yields a contract-violatingRawMessageStartEvent(message=None), which leaks to the consumer as an extra stream event.Any consumer that trusts the type annotations (for example
event.message.usage, as in the original report via pydantic-ai's streaming path onAsyncAnthropicBedrock) then crashes withAttributeError: 'NoneType' object has no attribute 'usage'. This is the same failure reported in #1647, still reproducible onmain.Reproduction (end to end, offline)
Feeding a complete Messages stream plus the metrics trailer through the real
AnthropicBedrockclient (binary eventstream frames, decoder, rawmessages.create(stream=True)):Fix
In
lib/bedrock/_stream_decoder.py(hand-maintained, not Stainless-generated),_chunk_bytes_to_ssenow drops the metrics-only trailer instead of forwarding it:completionfield) still route asevent="completion"(unchanged)typeand nocompletionfield, the metrics-only trailer, is dropped so it never reaches the stream-event unionTests
Adds two regression tests to
tests/lib/test_bedrock.py:test_chunk_bytes_to_sse_drops_metrics_only_trailerasserts the decoder returnsNonefor the trailertest_metrics_trailer_would_violate_stream_event_contractdocuments the union fallback the drop protects againstFull
tests/lib/test_bedrock.pypasses, andruffandpyrightare clean on the changed files.Fixes #1647.