Haystack

Use Eden AI with Haystack (deepset) to build LLM and RAG pipelines on 500+ AI models through one EU-hosted API key.

Overview

Haystack is deepset’s open-source Python framework for building production LLM applications, RAG pipelines and agents. Eden AI ships as an official Haystack integration (the edenai-haystack package), so your pipelines reach models from OpenAI, Anthropic, Google, Mistral and 500+ more behind one key, with EU-based, GDPR-aligned inference. The integration provides three components:

EdenAIChatGenerator for chat completion
EdenAITextEmbedder to embed a query string
EdenAIDocumentEmbedder to embed documents for indexing

Models use Eden AI’s provider/model naming convention, for example anthropic/claude-sonnet-4-5 or mistral/mistral-large-latest.

Installation

pip install edenai-haystack

uv add edenai-haystack

Quick Start

Set your key, then generate a chat response with the EdenAIChatGenerator:

EDENAI_API_KEY=your_api_key_here

from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.edenai import EdenAIChatGenerator

generator = EdenAIChatGenerator(model="mistral/mistral-large-latest")

response = generator.run([ChatMessage.from_user("What is Eden AI?")])
print(response["replies"][0].text)

The generator reads your key from the EDENAI_API_KEY environment variable by default.

Switching models

Change providers by changing the model string. Every Eden AI model is reachable the same way:

from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.edenai import EdenAIChatGenerator

messages = [ChatMessage.from_user("Write a haiku about the sea.")]

for model in [
    "openai/gpt-4o-mini",
    "anthropic/claude-sonnet-4-5",
    "mistral/mistral-large-latest",
]:
    generator = EdenAIChatGenerator(model=model)
    print(model, "->", generator.run(messages)["replies"][0].text)

Streaming

Pass a streaming_callback to stream tokens as they are generated:

from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.edenai import EdenAIChatGenerator

generator = EdenAIChatGenerator(
    model="mistral/mistral-large-latest",
    streaming_callback=print_streaming_chunk,
)
generator.run([ChatMessage.from_user("Count from one to five.")])

Embeddings and RAG

Use EdenAITextEmbedder to embed a query and EdenAIDocumentEmbedder to embed documents, so you can build a fully sovereign RAG stack (retrieval and generation) on EU-hosted models:

from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack_integrations.components.embedders.edenai import (
    EdenAIDocumentEmbedder,
    EdenAITextEmbedder,
)

document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")

# Index documents with their embeddings
documents = [Document(content="Eden AI is a unified API for 500+ AI models.")]
doc_embedder = EdenAIDocumentEmbedder(model="openai/text-embedding-3-small")
document_store.write_documents(doc_embedder.run(documents)["documents"])

# Embed the query and retrieve
text_embedder = EdenAITextEmbedder(model="openai/text-embedding-3-small")
retriever = InMemoryEmbeddingRetriever(document_store=document_store)
query_embedding = text_embedder.run("What is Eden AI?")["embedding"]
print(retriever.run(query_embedding=query_embedding)["documents"])

Available Models

Use the provider/model format for any Eden AI model: Chat

openai/gpt-4o-mini
anthropic/claude-sonnet-4-5
mistral/mistral-large-latest
google/gemini-2.5-flash

Embeddings

openai/text-embedding-3-small
mistral/mistral-embed

Browse the full list in the Eden AI models catalog.

Environment Variables

EDENAI_API_KEY=your_api_key_here

Next Steps

Chat Completions - Core LLM endpoint
List LLM Models - Browse available providers and models
aisuite - Another unified LLM interface
LangChain - Build LLM apps with LangChain

V3 Documentation

Quick Start

Overview

LLMs

Expert Models

General

Organization

Data Governance

Integrations

Overview

Installation

Quick Start

Switching models

Streaming

Embeddings and RAG

Available Models

Environment Variables

Next Steps

​Overview

​Installation

​Quick Start

​Switching models

​Streaming

​Embeddings and RAG

​Available Models

​Environment Variables

​Next Steps

Overview

Installation

Quick Start

Switching models

Streaming

Embeddings and RAG

Available Models

Environment Variables

Next Steps