v2.28.0

Upgrade Notes

• As part of the migration from requests to httpx, request_with_retry and async_request_with_retry (in haystack.utils.requests_utils) no longer raise requests.exceptions.RequestException on failure; they now raise httpx.HTTPError instead. This also affects HuggingFaceTEIRanker, which relies on these utilities. Users catching requests.exceptions.RequestException should update their code to catch httpx.HTTPError.

• The LLM component now requires user_prompt to be provided at initialization and it must contain at least one Jinja2 template variable (e.g. {{ variable_name }}). This ensures the component always exposes at least one required input socket, which is necessary for correct pipeline scheduling.

  required_variables now defaults to "*" (all variables in user_prompt are required), and passing an empty list raises a ValueError.

  If you are affected: update any code that instantiates LLM without a user_prompt, or with a user_prompt that has no template variables, to include at least one variable.

  Before:

  llm = LLM(chat_generator=OpenAIChatGenerator(), system_prompt="You are helpful.")

  After:

  llm = LLM(
      chat_generator=OpenAIChatGenerator(),
      system_prompt="You are helpful.",
      user_prompt='{% message role="user" %}{{ query }}{% endmessage %}',
  )

• Agent.run() and Agent.run_async() now require messages as an explicit argument (no longer optional). If you were relying on the default None value in Haystack version 2.26 or 2.27, pass an empty list instead:

  agent.run(messages=[], ...)

  LLM.run() and LLM.run_async() are unaffected — they still accept None and default to an empty list internally.

New Features

• Tools and components can now declare a State (or State | None) parameter in their signature to receive the live agent State object at invocation time — no extra wiring needed.

  For function-based tools created with @tool or create_tool_from_function, add a state parameter annotated as State:

  from haystack.components.agents import State
  from haystack.tools import tool
  
  @tool
  def my_tool(query: str, state: State) -> str:
      """Search using context from agent state."""
      history = state.get("history")
      ...

  For component-based tools created with ComponentTool, declare a State input socket on the component's run method:

  from haystack import component
  from haystack.components.agents import State
  from haystack.tools import ComponentTool
  
  @component
  class MyComponent:
      @component.output_types(result=str)
      def run(self, query: str, state: State) -> dict:
          history = state.get("history")
          ...
  
  tool = ComponentTool(component=MyComponent())

  In both cases ToolInvoker automatically injects the runtime State object before calling the tool, and State/Optional[State] parameters are excluded from the LLM-facing schema so the model is not asked to supply them.

  This is an alternative to the existing inputs_from_state and outputs_to_state options on Tool and ComponentTool, which map individual state keys to specific tool parameters and outputs declaratively. Injecting the full State object is more flexible and useful when a tool needs to read from or write to multiple keys, but it couples the tool implementation directly to State.

Enhancement Notes

• Clarify in the Markdown-producing converter documentation that DocumentCleaner with its default settings can flatten Markdown output, and update the example pipelines for PaddleOCRVLDocumentConverter, MistralOCRDocumentConverter, AzureDocumentIntelligenceConverter, and MarkItDownConverter to avoid routing Markdown content through the default cleaner configuration.
• Made _create_agent_snapshot robust towards serialization errors. If serializing agent component inputs fails, a warning is logged and an empty dictionary is used as a fallback, preventing the serialization error from masking the real pipeline runtime error.
• Standardize HTTP request handling in Haystack by adopting httpx for both synchronous and asynchronous requests, replacing requests. Error reporting for failed requests has also been improved: exceptions now include additional details alongside the reason field.
• Add run_async method to LLMMetadataExtractor. ChatGenerator requests now run concurrently using the existing max_workers init parameter.
• MarkdownHeaderSplitter now accepts a header_split_levels parameter (list of integers 1–6, default all levels) to control which header depths create split boundaries. For example, header_split_levels=[1, 2] splits only on # and ## headers, merging content under deeper headers into the preceding chunk.
• MarkdownHeaderSplitter now ignores # lines that appear inside fenced code blocks (triple-backtick or triple-tilde), preventing Python comments and other hash-prefixed lines in code from being misidentified as Markdown headers.
• Expand the PaddleOCRVLDocumentConverter documentation with more detailed guidance on advanced parameters, common usage scenarios, and a more realistic configuration example for layout-heavy documents.

Bug Fixes

• Fix ToolInvoker._merge_tool_outputs silently appending None to list-typed state when a tool's outputs_to_state source key is absent from the tool result. This is a common scenario with PipelineTool wrapping a pipeline that has conditional branches where not all outputs are always produced even if defined in outputs_to_state. The mapping is now skipped entirely when the source key is not present in the result dict.

• When using the MarkdownHeaderSplitter, in the split chunks, the child header previously lost its direct parent header in the metadata. Previously if one executed the code below:

  from haystack.components.preprocessors import MarkdownHeaderSplitter
  from haystack import Document
  text = """
  # header 1
  intro text
  
  ## header 1.1
  text 1
  
  ## header 1.2
  text 2
  
  ### header 1.2.1
  text 3
  
  ### header 1.2.2
  text 4
  """
  
  document = Document(content=text)
  
  splitter = MarkdownHeaderSplitter(
          keep_headers=True,
          secondary_split="word"
  )
  result = splitter.run(documents=[document])["documents"]
  
  for doc in result:
      print(f"Header: {doc.meta['header']}, parent headers: {doc.meta['parent_headers']}")

  We would have expected this output:

  Header: header 1, parent headers: []
  Header: header 1.1, parent headers: ['header 1']
  Header: header 1.2, parent headers: ['header 1']
  Header: header 1.2.1, parent headers: ['header 1', 'header 1.2']
  Header: header 1.2.2, parent headers: ['header 1', 'header 1.2']
  

  But instead we actually got:

  Header: header 1, parent headers: []
  Header: header 1.1, parent headers: []
  Header: header 1.2, parent headers: ['header 1']
  Header: header 1.2.1, parent headers: ['header 1']
  Header: header 1.2.2, parent headers: ['header 1', 'header 1.2']
  

  The error happened when a parent header had its own content chunk before the first child header.

  This has been fixed so even when a parent header has its own content chunk before the first child header all content is preserved.

• Reverts the change that made Agent messages optional as it caused issues with pipeline execution. As a consequence, the LLM component now defaults to an empty messages list unless provided at runtime.

💙 Big thank you to everyone who contributed to this release!

@Aftabbs, @Amanbig, @anakin87, @bilgeyucel, @bogdankostic, @davidsbatista, @dina-deifallah, @jimmyzhuu, @julian-risch, @kacperlukawski, @maxdswain, @MechaCritter, @ritikraj2425, @sarahkiener, @sjrl, @soheinze, @srini047, @tholor

v2.28.0-rc2

v2.28.0-rc2

v2.28.0-rc1

v2.28.0-rc1

v2.27.0

⭐️ Highlights

🔌 Automatic List Joining in Pipeline

When a component expects a list as input, pipelines now automatically join multiple inputs into that list (no extra components needed), even if they come in different but compatible types. This enables patterns like combining a plain query string with a list of ChatMessage objects into a single list[ChatMessage] input.

Supported conversations:

Source Types	Target Type	Behavior
T + T	list[T]	Combines multiple inputs into a list of the same type.
T + list[T]	list[T]	Merges single items and lists into a single list.
str + ChatMessage	list[str]	Converts all inputs to str and combines into a list.
str + ChatMessage	list[ChatMessage]	Converts all inputs to ChatMessage and combines into a list.

Learn more about how to simplify list joins in pipelines in 📖 Smart Pipeline Connections: Implicit List Joining

🗄️ Better Developer Experience for DocumentStores

The metadata inspection and filtering utilities (count_documents_by_filter, count_unique_metadata_by_filter, get_metadata_field_min_max, etc.) are now available in the InMemoryDocumentStore, aligning it with other document stores.

You can prototype locally in memory and easily debug, filter, and inspect the data in the document store during development, then reuse the same logic in production. See all available methods in InMemoryDocumentStore API reference.

🚀 New Features

• Added new operations to the InMemoryDocumentStore: count_documents_by_filter, count_unique_metadata_by_filter, get_metadata_fields_info, get_metadata_field_min_max, get_metadata_field_unique_values

• AzureOpenAIChatGenerator now exposes a SUPPORTED_MODELS class variable listing supported model IDs, for example gpt-5-mini and gpt-4o. To view all supported models go to the API reference or run:

  from haystack.components.generators.chat import AzureOpenAIChatGenerator
  print(AzureOpenAIChatGenerator.SUPPORTED_MODELS)

  We will add this for other model providers in their respective ChatGenerator components step by step.

• Added partial support for the image-text-to-text task in HuggingFaceLocalChatGenerator.

  This allows the use of multimodal models like Qwen 3.5 or Ministral with text-only inputs. Complete multimodal support via Hugging Face Transformers might be addressed in the future.

• Added async filter helpers to the InMemoryDocumentStore: update_by_filter_async(), count_documents_by_filter_async(), and count_unique_metadata_by_filter_async().

⚡️ Enhancement Notes

• Add async variants of metadata methods to InMemoryDocumentStore: get_metadata_fields_info_async(), get_metadata_field_min_max_async(), and get_metadata_field_unique_values_async(). These rely on the store's thread-pool executor, consistent with the existing async method pattern.
• Add _to_trace_dict method to ImageContent and FileContent dataclasses. When tracing is enabled, the large base64-encoded binary fields (base64_image and base64_data) are replaced with placeholder strings (e.g. "Base64 string (N characters)"), consistent with the behavior of ByteStream._to_trace_dict.
• Pipelines now support auto-variadic connections with type conversion. When multiple senders are connected to a single list-typed input socket, the senders no longer need to all produce the exact same type since compatible conversions are applied per edge. Supported scenarios include T + T -> list[T], T + list[T] -> list[T], str + ChatMessage -> list[str], str + ChatMessage -> list[ChatMessage], and all other str <-> ChatMessage conversion variants. This enables pipeline patterns like joining a plain query string with a list of ChatMessage objects into a single list[ChatMessage] input without any extra components.

🔒 Security Notes

• Fixed an issue in ChatPromptBuilder where specially crafted template variables could be interpreted as structured content (e.g., images, tool calls) instead of plain text.

  Template variables are now automatically sanitized during rendering, ensuring they are always treated as plain text.

🐛 Bug Fixes

• Fix malformed log format string in DocumentCleaner. The warning for documents with None content used %{document_id} instead of {document_id}, preventing proper interpolation of the document ID.
• Fix ToolInvoker._merge_tool_outputs silently appending None to list-typed state when a tool's outputs_to_state source key is absent from the tool result. This is a common scenario with PipelineTool wrapping a pipeline that has conditional branches where not all outputs are always produced even if defined in outputs_to_state. The mapping is now skipped entirely when the source key is not present in the result dict.
• Fixed an off-by-one error in InMemoryDocumentStore.write_documents that caused the BM25 average document length to be systematically underestimated.
• Resolve $defs/$ref in tool parameter schemas before sending them to the HuggingFace API. The HuggingFace API does not support JSON Schema $defs references, which are generated by Pydantic when tool parameters contain dataclass types. This fix inlines all $ref pointers and removes the $defs section from tool schemas in HuggingFaceAPIChatGenerator.
• The default bm25_tokenization_regex in InMemoryDocumentStore now uses r"(?u)\b\w+\b", including single-character words (e.g., "a", "C") in BM25 scoring. Previously, the regex r"(?u)\b\w\w+\b" excluded these tokens. This change may slightly alter retrieval results. To restore the old behavior, explicitly pass the previous regex when initializing the document store.

💙 Big thank you to everyone who contributed to this release!

@aayushbaluni, @anakin87, @bilgeyucel, @bogdankostic, @Br1an67, @ComeOnOliver, @davidsbatista, @jnMetaCode, @julian-risch, @Krishnachaitanyakc, @maxdswain, @pandego, @RMartinWhozfoxy, @satishkc7, @sjrl, @srini047, @SyedShahmeerAli12, @v-tan, @xr843

v2.27.0-rc1

v2.27.0-rc1

v2.26.1

Security Notes

• Fixed an issue in ChatPromptBuilder where specially crafted template variables could be interpreted as structured content (e.g., images, tool calls) instead of plain text. Template variables are now automatically sanitized during rendering, ensuring they are always treated as plain text.

🐛 Bug Fixes

• Fix malformed log format string in DocumentCleaner. The warning for documents with None content used %{document_id} instead of {document_id}, preventing proper interpolation of the document ID.

v2.26.1-rc1

v2.26.1-rc1

v2.26.0

⭐ Highlights

🧠 More Flexible Agents with Dynamic System Prompts

Agent now supports Jinja2 templating in system_prompt, enabling runtime parameter injection and conditional logic directly in system messages. This makes it easier to adapt agent behavior dynamically (e.g. language, tone, time-aware responses) and reuse agents across contexts without redefining prompts

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

agent = Agent(
    chat_generator=OpenAIChatGenerator(),
    tools=[weather_tool],
    system_prompt="""{% message role='system' %}
    You always respond in {{language}}.
    {% endmessage %}""",
    required_variables=["language"],
)

result = agent.run(
    messages=[ChatMessage.from_user("What is the weather in London?")], 
    language="Italian" # required variable for the system prompt
)
print(result["last_message"].text)
# >> Il tempo a Londra è soleggiato.

🔍 LLMRanker for LLM-Based Reranking

LLMRanker introduces LLM-powered reranking, treating relevance as a semantic reasoning task rather than similarity scoring. This can yield better results for complex or multi-step queries compared to cross-encoders. The component can also filter out irrelevant/duplicate documents entirely, helping provide higher-quality context in RAG pipelines and agent workflows while keeping context windows lean.

from haystack import Document
from haystack.components.rankers import LLMRanker

ranker = LLMRanker()

documents = [
    Document(id="paris", content="Paris is the capital of France."),
    Document(id="berlin", content="Berlin is the capital of Germany."),
]

result = ranker.run(query="capital of Germany", documents=documents)
print(result["documents"][0].id)  # "berlin"

📦 Discover Supported Models Programmatically

OpenAIChatGenerator, OpenAIResponsesChatGenerator, and AzureOpenAIResponsesChatGenerator now expose a SUPPORTED_MODELS class variable, giving you the list of models supported by each component.

from haystack.components.generators.chat import OpenAIChatGenerator
print(OpenAIChatGenerator.SUPPORTED_MODELS)

Note: We’ll roll out this pattern to other ChatGenerator components step by step. See issue #10627 for progress.

🚀 New Features

• Added LLMRanker, a new ranker component that uses a ChatGenerator and PromptBuilder to rerank documents based on JSON-formatted LLM output. LLMRanker supports configurable prompts, optional custom chat generators, runtime top_k overrides, and serialization.

• AzureOpenAIResponsesChatGenerator exposes a SUPPORTED_MODELS class variable listing supported model IDs, for example gpt-5-mini and gpt-4o. To view all supported models go to the [API reference](https://docs.haystack.deepset.ai/reference/generators-api#azureopenairesponseschatgenerator) or run:

  from haystack.components.generators.chat import AzureOpenAIResponsesChatGenerator
  print(AzureOpenAIResponsesChatGenerator.SUPPORTED_MODELS)

• We now allow a component's whose input type is typed as a union of lists (e.g. list[str] | list[ChatMessage]) to allow multiple input connections. Previously we only supported bare lists (e.g. list[str]) or optional lists (e.g. list[str] | None) to allow multiple input connections. A common use case for this is using the AnswerBuilder component which has it's replies input typed as list[str] | list[ChatMessage].

• The system_prompt initialization parameter of the Agent component now supports Jinja2 message template syntax. This allows you to define the template at initialization time and pass runtime variables when calling the run method. This can be useful to inject dynamic values (such as the current time) or to add conditional instructions.

  Example usage:

  from haystack.components.agents import Agent
  from haystack.components.generators.chat import OpenAIChatGenerator
  from haystack.dataclasses import ChatMessage
  from haystack.tools import tool
  
  @tool
  def weather(location: str) -> str:
      return f"The weather in {location} is sunny."
  
  agent = Agent(
      chat_generator=OpenAIChatGenerator(),
      tools=[weather],
      system_prompt="""{% message role='system' %}
      You always respond in {{language}}.
      {% endmessage %}""",
      required_variables=["language"],
  )
  
  messages = [ChatMessage.from_user("What is the weather in London?")]
  
  result = agent.run(messages=messages, language="Italian")
  print(result["last_message"].text)
  # >> Il tempo a Londra è soleggiato.

• OpenAIChatGenerator exposes a SUPPORTED_MODELS class variable listing supported model IDs, for example gpt-5-mini and gpt-4o. To view all supported models go to the [API reference](https://docs.haystack.deepset.ai/reference/generators-api#openaichatgenerator) or run:

  from haystack.components.generators.chat import OpenAIChatGenerator
  print(OpenAIChatGenerator.SUPPORTED_MODELS)

⚡️Enhancement Notes

• SearchableToolset now supports customizing the bootstrap search tool's name, description, and parameter descriptions via three new optional __init__ parameters: search_tool_name, search_tool_description, and search_tool_parameters_description. This allows users to tune the LLM-facing metadata to work better with different models.

  Example usage:

  from haystack.tools import SearchableToolset
  
  toolset = SearchableToolset(
      catalog=my_tools,
      search_tool_name="find_tools",
      search_tool_description="Find tools by keyword. Pass 1-3 words, not sentences.",
      search_tool_parameters_description={
          "tool_keywords": "Single words only, e.g. 'hotel booking'.",
      },
  )

• Add support for python 3.14 to Haystack. Haystack already mostly supported python 3.14. Only minor changes were needed in regards to our type serialization and type checking when handling bare Union types.

• Make the runtime parameter messages to Agent messages optional since it is possible to execute the agent with only providing a user_prompt.

• Improve performance of HuggingFaceAPIDocumentEmbedder.run_async by requesting embedding inference concurrently. This can be controlled using the new concurrency_limit parameter.

• Removed redundant deepcopy operations from Pipeline and AsyncPipeline execution. Component outputs are no longer deepcopied when collecting pipeline results, as inputs are already deepcopied before each component executes, preventing unintended mutations. Component inputs and outputs are also no longer deepcopied before being stored in tracing spans. These changes improve pipeline execution performance, especially when large objects (e.g., lists of Documents) flow between components and when include_outputs_from is used.

• Reduced unnecessary deepcopies in Agent for improved performance. Replaced deepcopy of state_schema with a shallow dict copy since only top-level keys are modified, and removed deepcopy of agent_inputs for span tags since the dict is freshly created and only used for tracing.

• Enable async embedding-based splitting with a new run_async method on EmbeddingBasedDocumentSplitter.

• Added gpt-5.4 to OpenAIChatGenerator's list of supported models.

• Added runtime validation of component output keys in Pipeline and AsyncPipeline. When a component returns keys that were not declared in its @component.output_types, the pipeline now logs a warning identifying the misconfigured component. This helps diagnose issues where a component returns unexpected keys, which previously caused a confusing "Pipeline Blocked" error pointing to an unexpected (downstream) component.

🐛 Bug Fixes

• Fixed Agent.run_async to mirror Agent.run crash handling for internal chat_generator and tool_invoker failures. Async runs now wrap internal PipelineRuntimeError exceptions with Agent context and attach pipeline snapshots so standalone async failures can be debugged and resumed consistently.

• Fixed ToolBreakpoint validation in Agent.run and Agent.run_async to validate against tools selected for the current run. This allows breakpoints for runtime tool overrides to work correctly.

• Replaced in-place dataclass attribute mutation with dataclasses.replace() across multiple components to prevent unintended side-effects when the same dataclass instance is shared across pipeline branches.

  Affected components and dataclasses:

  • ChatPromptBuilder and DynamicChatPromptBuilder: ChatMessage._content
  • HuggingFaceLocalChatGenerator: ChatMessage._content
  • HuggingFaceTEIRanker: Document.score
  • MetaFieldRanker: Document.score
  • SentenceTransformersSimilarityRanker: Document.score
  • TransformersSimilarityRanker: Document.score
  • ExtractiveReader: ExtractedAnswer.query
  • InMemoryDocumentStore: Document.embedding

• Update Pipeline.inputs() to return any variadic inputs as not mandatory if they already have a connection. Removed the utility functions describe_pipeline_inputs and describe_pipeline_inputs_as_string from haystack/core/pipeline/descriptions.py since they were not used and not referenced in the documentation. Use the Pipeline.inputs() method to inspect the inputs of a pipeline.

• Fixed a bug in the pipeline scheduling logic where a component with all-optional inputs (e.g. Agent after making messages optional) could be scheduled ahead of a variadic joiner (e.g. ListJoiner) that was still waiting on inputs. The fix updates the tiebreaking logic in `_tiebreak_wai...

v2.26.0-rc1

v2.26.0-rc1

v2.25.2

🐛 Bug Fixes

• Reverts the change that made Agent messages optional as it caused issues with pipeline execution. As a consequence, the LLM component now defaults to an empty messages list unless provided at runtime.