v2.10.0

⭐️ Highlights

Improved Pipeline.run() Logic

The new Pipeline.run() logic fixes common pipeline issues, including exceptions, incorrect component execution, missing intermediate outputs, and premature execution of lazy variadic components. While most pipelines should remain unaffected, we recommend carefully reviewing your pipeline executions if you are using cyclic pipelines or pipelines with lazy variadic components to ensure their behavior has not changed. You can use this tool to compare the execution traces of your pipeline with the old and new logic.

AsyncPipeline for Async Execution

Together with the new Pipeline.run logic, AsyncPipeline enables asynchronous execution, allowing pipeline components to run concurrently whenever possible. This leads to significant speed improvements, especially for pipelines processing data in parallel branches such as hybrid retrieval setting.

Source Codes

Hybrid Retrieval

hybrid_rag_retrieval = AsyncPipeline()
hybrid_rag_retrieval.add_component("text_embedder", SentenceTransformersTextEmbedder())
hybrid_rag_retrieval.add_component("embedding_retriever", InMemoryEmbeddingRetriever(document_store=document_store))
hybrid_rag_retrieval.add_component("bm25_retriever", InMemoryBM25Retriever(document_store=document_store))

hybrid_rag_retrieval.connect("text_embedder", "embedding_retriever")
hybrid_rag_retrieval.connect("bm25_retriever", "document_joiner")
hybrid_rag_retrieval.connect("embedding_retriever", "document_joiner")

async def run_inner():
    return await hybrid_rag_retrieval.run({
      "text_embedder": {"text": query}, 
      "bm25_retriever": {"query": query}
      })

results = asyncio.run(run_inner())

Parallel Translation Pipeline

from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack import AsyncPipeline
from haystack.utils import Secret

# Create prompt builders with templates at initialization
spanish_prompt_builder = ChatPromptBuilder(template="Translate this message to Spanish: {{user_message}}")
turkish_prompt_builder = ChatPromptBuilder(template="Translate this message to Turkish: {{user_message}}")
thai_prompt_builder = ChatPromptBuilder(template="Translate this message to Thai: {{user_message}}")

# Create LLM instances
spanish_llm = OpenAIChatGenerator()
turkish_llm = OpenAIChatGenerator()
thai_llm = OpenAIChatGenerator()

# Create and configure pipeline
pipe = AsyncPipeline()

# Add components
pipe.add_component("spanish_prompt_builder", spanish_prompt_builder)
pipe.add_component("turkish_prompt_builder", turkish_prompt_builder)
pipe.add_component("thai_prompt_builder", thai_prompt_builder)

pipe.add_component("spanish_llm", spanish_llm)
pipe.add_component("turkish_llm", turkish_llm)
pipe.add_component("thai_llm", thai_llm)

# Connect components
pipe.connect("spanish_prompt_builder.prompt", "spanish_llm.messages")
pipe.connect("turkish_prompt_builder.prompt", "turkish_llm.messages")
pipe.connect("thai_prompt_builder.prompt", "thai_llm.messages")

user_message = """
In computer programming, the async/await pattern is a syntactic feature of many programming languages that 
allows an asynchronous, non-blocking function to be structured in a way similar to an ordinary synchronous function. 
It is semantically related to the concept of a coroutine and is often implemented using similar techniques, 
and is primarily intended to provide opportunities for the program to execute other code while waiting 
for a long-running, asynchronous task to complete, usually represented by promises or similar data structures.
"""

# Run the pipeline with simplified input
res = pipe.run(data={"user_message": user_message})

# Print results
print("Spanish translation:", res["spanish_llm"]["generated_messages"][0].text)
print("Turkish translation:", res["turkish_llm"]["generated_messages"][0].text)
print("Thai translation:", res["thai_llm"]["generated_messages"][0].text)

Tool Calling Support Everywhere

Tool calling is now universally supported across all chat generators, making it easier than ever for developers to port tools across different platforms. Simply switch the chat generator used, and tooling will work seamlessly without any additional configuration. This update applies across AzureOpenAIChatGenerator, HuggingFaceLocalChatGenerator, and all core integrations, including AnthropicChatGenerator, CohereChatGenerator, AmazonBedrockChatGenerator, and VertexAIGeminiChatGenerator. With this enhancement, tool usage becomes a native capability across the ecosystem, enabling more advanced and interactive agentic applications.

Visualize Your Pipelines Locally

Pipeline visualization is now more flexible, allowing users to render pipeline graphs locally without requiring an internet connection or sending data to an external service. By running a local Mermaid server with Docker, you can generate visual representations of your pipelines using draw() or show(). Learn more in Visualizing Pipelines

New Components for Smarter Document Processing

This release introduces new components that enhance document processing capabilities. CSVDocumentSplitter and CSVDocumentCleaner make handling CSV files more efficient. LLMMetadaExtractor leverages an LLM to analyze documents and enrich them with relevant metadata, improving searchability and retrieval accuracy.

⬆️ Upgrade Notes

• The DOCXToDocument converter now returns a Document object with DOCX metadata stored in the meta field as a dictionary under the key docx. Previously, the metadata was represented as a DOCXMetadata dataclass. This change does not impact reading from or writing to a Document Store.
• Removed the deprecated NLTKDocumentSplitter, it's functionalities are now supported by the DocumentSplitter.
• The deprecated FUNCTION role has been removed from the ChatRole enum. Use TOOL instead. The deprecated class method ChatMessage.from_function has been removed. Use ChatMessage.from_tool instead.

🚀 New Features

• Added a new component ListJoiner which joins lists of values from different components to a single list.

• Introduced the OpenAPIConnector component, enabling direct invocation of REST endpoints as specified in an OpenAPI specification. This component is designed for direct REST endpoint invocation without LLM-generated payloads, users needs to pass the run parameters explicitly. Example:

  from haystack.utils import Secret 
  from haystack.components.connectors.openapi import OpenAPIConnector  
  
  connector = OpenAPIConnector(openapi_spec="https://bit.ly/serperdev_openapi", credentials=Secret.from_env_var("SERPERDEV_API_KEY")) 
  response = connector.run(operation_id="search", parameters={"q": "Who was Nikola Tesla?"} ) 

• Adding a new component, LLMMetadaExtractor, which can be used in an indexing pipeline to extract metadata from documents based on a user-given prompt and return the documents with the metadata field with the output of the LLM.

• Introduced CSVDocumentCleaner component for cleaning CSV documents.

  • Removes empty rows and columns, while preserving specified ignored rows and columns.
  • Customizable number of rows and columns to ignore during processing.

• Introducing CSVDocumentSplitter: The CSVDocumentSplitter splits CSV documents into structured sub-tables by recursively splitting by empty rows and columns larger than a specified threshold. This is particularly useful when converting Excel files which can often have multiple tables within one sheet.

⚡️ Enhancement Notes

• Enhanced SentenceTransformersDocumentEmbedder and SentenceTransformersTextEmbedder to accept an additional parameter, which is passed directly to the underlying SentenceTransformer.encode method for greater flexibility in embedding customization.
• Added completion_start_time metadata to track time-to-first-token (TTFT) in streaming responses from Hugging Face API and OpenAI (Azure).
• Enhancements to Date Filtering in MetadataRouter:
  • Improved date parsing in filter utilities by introducing _parse_date, which first attempts datetime.fromisoformat(value) for backward compatibility and then falls back to dateutil.parser.parse() for broader ISO 8601 support.
  • Resolved a common issue where comparing naive and timezone-aware datetimes resulted in TypeError. Added _ensure_both_dates_naive_or_aware, which ensures both datetimes are either naive or aware. If one is missing a timezone, it is assigned the timezone of the other for consistency.
• When Pipeline.from_dict receives an invalid type (e.g. empty string), an informative PipelineError is now raised.
• Add jsonschema library as a core dependency. It is used in Tool and JsonSchemaValidator.
• Streaming callback run param support for HF chat generators.
• For the CSVDocumentCleaner, added remove_empty_rows & remove_empty_columns to optionally remove rows and columns. Also added keep_id to optionally allow for keeping the original document ID.
• Enhanced OpenAPIServiceConnector to support and be compatible with the new ChatMessage format.
• Updated Document's meta data after initializing the Document in DocumentSplitter as requested in issue #8741

⚠️ Deprecation Notes

• The ExtractedTableAnswer dataclass and the dataframe field in the Document dataclass are deprecated and will be removed in Haystack 2.11.0. Check out the GitHub discussion for motivation and details.

🐛 Bug Fixes

• Fixes a bug that causes pyright type checker to fail for all component objects.
• Haystack pipelines with Mermaid graphs are now compressed to reduce the size of the encoded base64 and avoid HTTP 400 errors when the graph is too large.
• The DOCXToDocument component now skips comment blocks in DOCX files that previously caused errors.
• Callable deserialization now works for all fully qualified import paths.
• Fix error messages for Document Classifier components, that suggested using nonexistent components for text classification.
• Fixed JSONConverter to properly skip converting JSON files that are not utf-8 encoded.
• • acyclic pipelines with multiple lazy variadic components not running all components
  • cyclic pipelines not passing intermediate outputs to components outside the cycle
  • cyclic pipelines with two or more optional or greedy variadic edges showing unexpected execution behavior
  • cyclic pipelines with two cycles sharing an edge raising errors
• Updated PDFMinerToDocument convert function to to double new lines between container_text so that passages can later by DocumentSplitter.
• In the Hugging Face API embedders, the InferenceClient.feature_extraction method is now used instead of InferenceClient.post to compute embeddings. This ensures a more robust and future-proof implementation.
• Improved OpenAIChatGenerator streaming response tool call processing: The logic now scans all chunks to correctly identify the first chunk with tool calls, ensuring accurate payload construction and preventing errors when tool call data isn't confined to the initial chunk.