How to Integrate Haystack for banking with Elasticsearch for startups

Combining Haystack for banking with Elasticsearch gives you a practical retrieval layer for agentic workflows in finance. Haystack handles orchestration, prompt pipelines, and document processing, while Elasticsearch gives you fast keyword + vector search over customer records, policy docs, transaction notes, or product knowledge.

For startups, this is the difference between a demo agent and a system that can answer regulated questions with traceable retrieval.

Prerequisites

• •Python 3.10+
• •An Elasticsearch cluster running locally or in the cloud
• •An API key or credentials for your Haystack for banking deployment
• •pip access to install Python packages
• •A document source to index:
  • •PDFs
  • •FAQ text
  • •internal policy docs
  • •customer support transcripts
• •Basic familiarity with:
  • •haystack
  • •elasticsearch
  • •embedding models

Install the packages:

pip install haystack-ai elasticsearch sentence-transformers

Integration Steps

1) Connect to Elasticsearch

Start by creating a client and verifying the cluster is reachable. In production, use TLS and API keys.

from elasticsearch import Elasticsearch

es = Elasticsearch(
    "https://localhost:9200",
    api_key=("your_api_key_id", "your_api_key_secret"),
)

print(es.info())

If you are using local development without auth, this is enough:

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")
print(es.ping())

2) Create an index for Haystack documents

Haystack works best when your documents are structured. Store text plus metadata such as source, customer segment, or compliance tags.

index_name = "banking_docs"

if not es.indices.exists(index=index_name):
    es.indices.create(
        index=index_name,
        mappings={
            "properties": {
                "content": {"type": "text"},
                "title": {"type": "text"},
                "source": {"type": "keyword"},
                "embedding": {
                    "type": "dense_vector",
                    "dims": 384,
                    "index": True,
                    "similarity": "cosine"
                }
            }
        },
    )

That mapping supports both full-text search and vector retrieval. For banking workflows, keep metadata fields explicit so you can filter by product line or document type later.

3) Write documents from Haystack into Elasticsearch

Use Haystack’s Document object to normalize your content before indexing. Then push the records into Elasticsearch with embeddings attached.

from haystack import Document
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

docs = [
    Document(
        content="A wire transfer over $10,000 requires additional verification.",
        meta={"title": "Wire Transfer Policy", "source": "compliance"}
    ),
    Document(
        content="Savings accounts accrue interest daily and post monthly.",
        meta={"title": "Savings Product FAQ", "source": "product"}
    ),
]

for doc in docs:
    embedding = model.encode(doc.content).tolist()

    es.index(
        index=index_name,
        document={
            "content": doc.content,
            "title": doc.meta["title"],
            "source": doc.meta["source"],
            "embedding": embedding,
        },
    )

es.indices.refresh(index=index_name)

This pattern keeps indexing simple and makes retrieval deterministic. If you already have a Haystack pipeline producing Document objects, insert this indexing step after parsing and chunking.

4) Retrieve relevant passages from Elasticsearch inside a Haystack pipeline

Now wire retrieval into your agent flow. In Haystack, a common pattern is: query -> retrieve -> generate. If your setup uses an external retriever component, keep Elasticsearch as the source of truth for retrieval.

from haystack import Pipeline, component
from haystack.dataclasses import ChatMessage

@component
class ElasticRetriever:
    @component.output_types(documents=list)
    def run(self, query: str):
        query_vec = model.encode(query).tolist()

        response = es.search(
            index=index_name,
            knn={
                "field": "embedding",
                "query_vector": query_vec,
                "k": 3,
                "num_candidates": 10,
            },
            _source=["content", "title", "source"],
        )

        documents = []
        for hit in response["hits"]["hits"]:
            documents.append(hit["_source"])

        return {"documents": documents}

pipeline = Pipeline()
pipeline.add_component("retriever", ElasticRetriever())

If your startup is using an LLM-backed agent, this retrieved context becomes the grounded input to the model. Keep the retriever narrow and deterministic; do not let the agent hallucinate over raw bank data.

5) Feed retrieved context into the answer generator

Take the top matches from Elasticsearch and pass them into your generation step. Here’s a minimal example that formats context for downstream prompting.

def build_context(docs):
    return "\n\n".join(
        f"Title: {d['title']}\nSource: {d['source']}\nContent: {d['content']}"
        for d in docs
    )

query = "Do large wire transfers need extra checks?"
result = pipeline.run({"retriever": {"query": query}})
context = build_context(result["retriever"]["documents"])

messages = [
    ChatMessage.from_system("Answer only from provided banking policy context."),
    ChatMessage.from_user(f"Question: {query}\n\nContext:\n{context}")
]

print(messages[1].content)

In a real agent system, replace the final print with your generator component or LLM call. The important part is that Elasticsearch controls retrieval and Haystack controls orchestration.

Testing the Integration

Run a simple end-to-end check: index one document, retrieve it with a query, and confirm the result contains the expected policy text.

test_query = "What happens when a wire transfer is above ten thousand dollars?"
response = es.search(
    index=index_name,
    knn={
        "field": "embedding",
        "query_vector": model.encode(test_query).tolist(),
        "k": 1,
        "num_candidates": 5,
    },
)

top_hit = response["hits"]["hits"][0]["_source"]
print(top_hit["title"])
print(top_hit["content"])

Expected output:

Wire Transfer Policy
A wire transfer over $10,000 requires additional verification.

If that returns correctly, your retrieval path is working. At that point you can plug it into your Haystack pipeline or banking agent without guessing whether search is broken.

Real-World Use Cases

• •

  Compliance Q&A agents

  • •Answer internal policy questions with citations from indexed banking procedures.
  • •Use metadata filters to restrict results by region, product, or regulatory scope.
• •

  Customer support copilots

  • •Retrieve relevant account servicing docs, fee schedules, and product FAQs.
  • •Ground responses in Elasticsearch-backed knowledge instead of model memory.
• •

  Operations assistants

  • •Search transaction investigation notes, KYC checklists, and escalation playbooks.
  • •Build workflows where Haystack routes tasks and Elasticsearch supplies evidence fast.

The pattern here is stable: Haystack manages the agent logic, Elasticsearch stores and retrieves evidence. For startups building banking agents, that split keeps your system auditable enough to ship.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.