Feat: Content Filtering Exception Handling #3634

AlanPonnachan · 2025-12-03T17:52:16Z

Unified Content Filtering Exception Handling

This PR closes #1035

Standardizing how content filtering events are handled across different model providers.

Previously, triggering a content filter resulted in inconsistent behaviors: generic ModelHTTPError (Azure), UnexpectedModelBehavior (Google), or silent failures depending on the provider. This PR introduces a dedicated exception hierarchy to allow users to catch and handle prompt refusals and response interruptions programmatically and consistently.

Key Changes:

New Exceptions: Added ContentFilterError (base), PromptContentFilterError (for input rejections, e.g., Azure 400), and ResponseContentFilterError (for output refusals).
OpenAI & Azure: Updated logic to raise PromptContentFilterError for Azure's specific 400 error body and ResponseContentFilterError when finish_reason='content_filter'.
Google Gemini: Updated _process_response to raise ResponseContentFilterError instead of UnexpectedModelBehavior when safety thresholds are triggered.
Anthropic: Added mapping for refusal stop reasons to raise ResponseContentFilterError.
Tests: Added comprehensive tests covering synchronous and streaming scenarios for OpenAI, Google, and Anthropic in tests/models/.

Example Usage:

from pydantic_ai import Agent
from pydantic_ai.exceptions import ContentFilterError

agent = Agent('openai:gpt-4o')

try:
    # If the prompt or generation triggers a safety filter
    await agent.run("Generate unsafe content...")
except ContentFilterError as e:
    # Catches both PromptContentFilterError and ResponseContentFilterError
    print(f"Request halted by safety filter: {e}")

rahim-figs · 2025-12-04T12:54:34Z

Is it possible to handle AWS Bedrock as well? Thanks.

dsfaccini

Hey @AlanPonnachan thank you for the PR! I've requested a couple small changes, let me know if you have any questions

pydantic_ai_slim/pydantic_ai/exceptions.py

pydantic_ai_slim/pydantic_ai/models/anthropic.py

pydantic_ai_slim/pydantic_ai/models/openai.py

tests/models/test_anthropic.py

tests/models/test_google.py

tests/models/test_openai.py

dsfaccini · 2025-12-04T19:25:57Z

one more thing I missed, please include the name of the new exception in the fallback model docs

ModelAPIError, which includes ModelHTTPError and ...

AlanPonnachan · 2025-12-05T11:06:46Z

@dsfaccini Thank you for the review. I’ve made the requested changes.

pydantic_ai_slim/pydantic_ai/models/openai.py

tests/models/test_openai.py

pydantic_ai_slim/pydantic_ai/exceptions.py

pydantic_ai_slim/pydantic_ai/models/openai.py

dsfaccini · 2025-12-05T16:31:33Z

hey @AlanPonnachan thanks a lot for your work! It looks very good now, I requested a couple more changes but once that's done and coverage passes I think the PR will be ready for merge.

AlanPonnachan · 2025-12-06T06:33:59Z

@dsfaccini Thanks again for the review! I’ve applied the requested changes. Test coverage is now at 100%.

DouweM

@AlanPonnachan Thanks for working on this! My main concern is that this doesn't actually make it consistent for all models that respond with finish_reason=='content_filter', just for Anthropic/Google/OpenAI.

DouweM · 2025-12-09T19:40:36Z

pydantic_ai_slim/pydantic_ai/exceptions.py

+        self.message = message
+
+
+class PromptContentFilterError(ContentFilterError):


Can we name this RequestContentFilterError for consistency with our ModelRequest/ModelResponse

DouweM · 2025-12-09T19:40:58Z

pydantic_ai_slim/pydantic_ai/exceptions.py

+    """Raised when the prompt triggers a content filter."""
+
+    def __init__(self, status_code: int, model_name: str, body: object | None = None):
+        message = f"Model '{model_name}' content filter was triggered by the user's prompt"


Or by the instructions, right? I'd prefer it to be a bit more generic

DouweM · 2025-12-09T19:46:47Z

pydantic_ai_slim/pydantic_ai/models/anthropic.py

            provider_details = {'finish_reason': raw_finish_reason}
            finish_reason = _FINISH_REASON_MAP.get(raw_finish_reason)
+            if finish_reason == 'content_filter':
+                raise ResponseContentFilterError(


I think we should do this in ModelRequestNode, so that it'll work for all models, not just Anthropic.

That would mean that it wouldn't trigger FallbackModel to try another model, but that will be possible once we do #3640.

DouweM · 2025-12-09T19:47:56Z

pydantic_ai_slim/pydantic_ai/models/google.py

            if finish_reason == 'content_filter' and raw_finish_reason:
-                raise UnexpectedModelBehavior(
-                    f'Content filter {raw_finish_reason.value!r} triggered', response.model_dump_json()
+                raise ResponseContentFilterError(


This is a breaking change for users that currently have except UnexpectedModelBehavior. So if we add a new exception, I think it should be a subclass of UnexpectedModelBehavior, not ModelAPIError (as it isn't really an API/HTTP/connection error).

Combined with the above, the solution may be to remove this check here, and to implement the same if response.finish_reason == 'content_filter' check in ModelRequestNode that will apply to all models.

If we want to get some details from the response up to that level, we can store them in response.provider_details

DouweM · 2025-12-09T19:51:01Z

pydantic_ai_slim/pydantic_ai/models/openai.py

+            if (error := body_dict.get('error')) and isinstance(error, dict):
+                error_dict = cast(dict[str, Any], error)
+                if error_dict.get('code') == 'content_filter':
+                    raise PromptContentFilterError(


What do you think about instead handling this by returning a ModelResponse with empty parts and finish_reason == 'content_filter', so that it can go through the same logic as the Anthropic and Google models?

I also don't think they distinguish between input/request and output/response content filter errors, so I don't think we need to here either.

AlanPonnachan and others added 2 commits December 3, 2025 17:35

add content filtering exceptions

4ac81d4

Merge branch 'main' into feature/content-filtering-exceptions

658f407

dsfaccini requested changes Dec 4, 2025

View reviewed changes

AlanPonnachan and others added 5 commits December 5, 2025 09:31

resolve issues

404a833

add docs

3771c79

Merge branch 'main' into feature/content-filtering-exceptions

28e20c8

test coverage

70bcb74

improve tests

e861a50

dsfaccini requested changes Dec 5, 2025

View reviewed changes

pydantic_ai_slim/pydantic_ai/models/openai.py Outdated Show resolved Hide resolved

tests/models/test_openai.py Outdated Show resolved Hide resolved

pydantic_ai_slim/pydantic_ai/exceptions.py Show resolved Hide resolved

pydantic_ai_slim/pydantic_ai/models/openai.py Show resolved Hide resolved

AlanPonnachan added 3 commits December 6, 2025 05:13

resolve issues

51506b2

test coverage

b67d216

fix ci fails

4ac1608

DouweM requested changes Dec 9, 2025

View reviewed changes

DouweM self-assigned this Dec 9, 2025

DouweM added the awaiting author revision label Dec 9, 2025

		self.message = message


		class PromptContentFilterError(ContentFilterError):

Feat: Content Filtering Exception Handling #3634

Are you sure you want to change the base?

Feat: Content Filtering Exception Handling #3634

Conversation

AlanPonnachan commented Dec 3, 2025

Unified Content Filtering Exception Handling

Uh oh!

rahim-figs commented Dec 4, 2025

Uh oh!

dsfaccini left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dsfaccini commented Dec 4, 2025

Uh oh!

AlanPonnachan commented Dec 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dsfaccini commented Dec 5, 2025

Uh oh!

AlanPonnachan commented Dec 6, 2025

Uh oh!

DouweM left a comment

Choose a reason for hiding this comment

Uh oh!

DouweM Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

DouweM Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

DouweM Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

DouweM Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

DouweM Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

DouweM Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants