How to Integrate Haystack for retail banking with Elasticsearch for multi-agent systems

Combining Haystack for retail banking with Elasticsearch gives you a practical pattern for agentic banking systems: Haystack handles retrieval, pipelines, and tool orchestration, while Elasticsearch gives you fast, indexed access to customer records, policy docs, transaction notes, and product knowledge. In a multi-agent setup, that means one agent can classify intent, another can retrieve account-specific context, and a third can generate compliant responses from grounded data.

Prerequisites

• •Python 3.10+
• •An Elasticsearch cluster running locally or in your environment
• •Access to the Haystack stack you’re using for retail banking
• •API keys or credentials for any bank-specific services behind your Haystack components
• •pip installed
• •Basic familiarity with:
  • •Haystack pipelines
  • •Elasticsearch indexing/search
  • •Python async/sync client usage

Install the core packages:

pip install haystack-ai elasticsearch

If your retail banking setup uses Haystack integrations for OpenAI, Hugging Face, or document stores, install those too.

Integration Steps

1) Connect to Elasticsearch

Start by creating an Elasticsearch client. In a banking agent system, this cluster usually stores customer support transcripts, product docs, policy pages, and retrieval-ready transaction metadata.

from elasticsearch import Elasticsearch

es = Elasticsearch(
    "http://localhost:9200",
    basic_auth=("elastic", "changeme"),
)

print(es.info())

If you’re running in production, use TLS and API keys instead of basic auth.

from elasticsearch import Elasticsearch

es = Elasticsearch(
    "https://your-es-host:9200",
    api_key="YOUR_API_KEY"
)

2) Create an index for banking knowledge

Use a dedicated index for retail banking content. Keep the schema simple and explicit so your agents can filter by document type, region, or product line.

index_name = "retail-banking-knowledge"

mapping = {
    "mappings": {
        "properties": {
            "doc_id": {"type": "keyword"},
            "title": {"type": "text"},
            "content": {"type": "text"},
            "product": {"type": "keyword"},
            "region": {"type": "keyword"},
            "updated_at": {"type": "date"}
        }
    }
}

if not es.indices.exists(index=index_name):
    es.indices.create(index=index_name, **mapping)

Now insert a few documents your agents can retrieve during conversations.

docs = [
    {
        "_index": index_name,
        "_id": "cc-fees-001",
        "_source": {
            "doc_id": "cc-fees-001",
            "title": "Credit Card Fee Policy",
            "content": "Annual fee is waived for premium customers. Late payment fee applies after grace period.",
            "product": "credit_card",
            "region": "us",
            "updated_at": "2026-01-01"
        }
    },
    {
        "_index": index_name,
        "_id": "loan-refi-001",
        "_source": {
            "doc_id": "loan-refi-001",
            "title": "Mortgage Refinance Eligibility",
            "content": "Customers must have at least 12 months of payment history and current LTV below 80%.",
            "product": "mortgage",
            "region": "us",
            "updated_at": "2026-01-03"
        }
    }
]

for doc in docs:
    es.index(**doc)

es.indices.refresh(index=index_name)

3) Wire Elasticsearch into Haystack retrieval

Haystack’s ElasticsearchDocumentStore is the cleanest bridge here. It lets your pipeline query Elasticsearch directly as the retriever’s backing store.

from haystack import Document
from haystack_integrations.document_stores.elasticsearch import ElasticsearchDocumentStore

document_store = ElasticsearchDocumentStore(
    hosts="http://localhost:9200",
    index=index_name,
    embedding_dim=768,
)

document_store.write_documents([
    Document(
        id="cc-fees-001",
        content="Annual fee is waived for premium customers. Late payment fee applies after grace period.",
        meta={"title": "Credit Card Fee Policy", "product": "credit_card", "region": "us"}
    ),
    Document(
        id="loan-refi-001",
        content="Customers must have at least 12 months of payment history and current LTV below 80%.",
        meta={"title": "Mortgage Refinance Eligibility", "product": "mortgage", "region": "us"}
    )
])

For semantic search in agent workflows, add an embedder and retriever. This is what lets one agent ask “What are the refinance rules?” and get grounded answers from indexed bank content.

from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers import EmbeddingRetriever

query_embedder = SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
retriever = EmbeddingRetriever(document_store=document_store)

query = query_embedder.run(text="What are the mortgage refinance requirements?")
results = retriever.run(query_embedding=query["embedding"])

for doc in results["documents"]:
    print(doc.meta["title"], doc.content)

4) Build a Haystack pipeline for multi-agent retrieval

In a multi-agent system, don’t let every agent hit Elasticsearch directly. Put retrieval behind a Haystack pipeline so you can control ranking, filtering, and prompt grounding in one place.

from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator

template = """
You are a retail banking assistant.
Answer only from the provided documents.

Question: {{question}}

Documents:
{% for doc in documents %}
- {{ doc.meta.title }}: {{ doc.content }}
{% endfor %}

Answer:
"""

pipe = Pipeline()
pipe.add_component("embedder", query_embedder)
pipe.add_component("retriever", retriever)
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", OpenAIGenerator(api_key="YOUR_OPENAI_API_KEY"))

pipe.connect("embedder.embedding", "retriever.query_embedding")
pipe.connect("retriever.documents", “prompt_builder.documents")
pipe.connect("prompt_builder.prompt", “llm.prompt")

A common pattern is to expose this pipeline as one tool inside an orchestrator agent. Another agent handles compliance checks; another handles customer intent classification; this retrieval pipeline supplies evidence.

5) Query it from an agent workflow

Here’s the practical call path: user question goes to an intent agent, then to the retrieval pipeline, then to the response generator.

result = pipe.run({
    “embedder”: {"text":"Can I refinance my mortgage?"},
    “prompt_builder”: {"question":"Can I refinance my mortgage?"}
})

print(result["llm"]["replies"][0].text)

If you want strict filtering by region or product before generation, apply filters at the document store layer or in the retriever configuration. That keeps US mortgage answers from pulling UK credit card policy text.

Testing the Integration

Run a direct retrieval test first. This verifies that Elasticsearch indexing and Haystack retrieval are both working before you wire in multiple agents.

test_query = query_embedder.run(text="Explain late payment fees")
test_results = retriever.run(query_embedding=test_query["embedding"])

assert len(test_results["documents"]) > 0

for d in test_results["documents"][:2]:
    print(f"{d.meta['title']} -> {d.content}")

Expected output:

Credit Card Fee Policy -> Annual fee is waived for premium customers. Late payment fee applies after grace period.

If that prints cleanly, your integration is good enough to plug into an orchestrated agent stack.

Real-World Use Cases

• •

  Customer support triage

  • •One agent classifies the request.
  • •Another retrieves policy-backed answers from Elasticsearch through Haystack.
  • •A final compliance agent checks whether the response stays within approved language.
• •

  Loan and card servicing assistants

  • •Retrieve eligibility rules, fee policies, repayment terms, and escalation paths.
  • •Keep responses grounded in indexed internal docs instead of model memory.
• •

  Back-office multi-agent workflows

  • •One agent searches case notes.
  • •Another summarizes account history.
  • •A third drafts next-step actions for human review using retrieved evidence from Elasticsearch-backed knowledge stores.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.