How to Integrate Haystack for healthcare with Elasticsearch for AI agents

Combining Haystack for healthcare with Elasticsearch gives you a practical retrieval layer for clinical AI agents. You get healthcare-aware document processing on one side, and fast hybrid search over large medical corpora on the other.

That matters when your agent needs to answer questions from discharge summaries, lab reports, policy docs, or clinical guidelines without scanning raw PDFs every time. The pattern is simple: ingest structured healthcare content with Haystack, index it in Elasticsearch, then let the agent retrieve grounded context before generating an answer.

Prerequisites

• •Python 3.10+
• •An Elasticsearch cluster running locally or in your VPC
• •elasticsearch Python client installed
• •Haystack installed with the healthcare components you plan to use
• •Access to healthcare documents in a supported format like PDF, TXT, JSON, or FHIR-derived text
• •Environment variables for Elasticsearch credentials if your cluster is secured

Install the core packages:

pip install haystack-ai elasticsearch

If your setup uses Haystack healthcare extensions, install those too:

pip install haystack-ai-haystack-healthcare

Integration Steps

1) Connect to Elasticsearch

Start by creating a client and verifying the cluster is reachable. In production, use TLS and API keys.

from elasticsearch import Elasticsearch

es = Elasticsearch(
    "https://localhost:9200",
    api_key=("elastic", "your-api-key")
)

print(es.info())

If you are using basic auth instead of API keys:

es = Elasticsearch(
    "https://localhost:9200",
    basic_auth=("elastic", "changeme"),
    verify_certs=True
)

2) Prepare healthcare documents with Haystack

Use Haystack to load and normalize documents before indexing. For healthcare workloads, keep metadata explicit: patient ID hashes, encounter dates, document type, and source system.

from haystack import Document

docs = [
    Document(
        content="Patient discharged with metformin 500mg twice daily. Follow-up in 2 weeks.",
        meta={
            "doc_type": "discharge_summary",
            "department": "internal_medicine",
            "patient_hash": "p_10492",
            "source": "ehr_export"
        }
    ),
    Document(
        content="HbA1c measured at 8.4%. Recommend diabetes management review.",
        meta={
            "doc_type": "lab_note",
            "department": "pathology",
            "patient_hash": "p_10492",
            "source": "lab_system"
        }
    )
]

If you are extracting from files, use a Haystack converter first, then attach healthcare metadata during preprocessing.

3) Create an Elasticsearch-backed document store

Haystack can write documents into Elasticsearch through its document store abstraction. This is the cleanest way to keep retrieval logic inside your pipeline.

from haystack_integrations.document_stores.elasticsearch import ElasticsearchDocumentStore

document_store = ElasticsearchDocumentStore(
    hosts=["https://localhost:9200"],
    basic_auth=("elastic", "changeme"),
    index="healthcare_docs",
    embedding_dim=384,
    similarity="cosine"
)

document_store.write_documents(docs)
print("Indexed:", document_store.count_documents())

For semantic search later, generate embeddings with a Haystack embedder and write them alongside the documents.

from haystack.components.embedders import SentenceTransformersDocumentEmbedder

embedder = SentenceTransformersDocumentEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
embedder.warm_up()

embedded_docs = embedder.run(documents=docs)["documents"]
document_store.write_documents(embedded_docs)

4) Build a retrieval pipeline for the agent

Now wire retrieval so your agent can pull relevant medical context before answering. This keeps responses grounded in indexed clinical text.

from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers import InMemoryEmbeddingRetriever

query_embedder = SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
retriever = InMemoryEmbeddingRetriever(document_store=document_store)

pipe = Pipeline()
pipe.add_component("query_embedder", query_embedder)
pipe.add_component("retriever", retriever)

pipe.connect("query_embedder.embedding", "retriever.query_embedding")

result = pipe.run({
    "query_embedder": {"text": "What follow-up was recommended after discharge?"}
})

for doc in result["retriever"]["documents"]:
    print(doc.content)

If you want hybrid retrieval in Elasticsearch itself, use BM25 plus embeddings at the index level depending on your version and mapping strategy. For many healthcare agent systems, that gives better recall on medication names and abbreviations.

5) Attach the retriever to your AI agent workflow

Your agent should call retrieval first, then answer from returned context only. That reduces hallucinations on clinical questions.

def retrieve_context(question: str):
    query_result = pipe.run({
        "query_embedder": {"text": question}
    })
    return [doc.content for doc in query_result["retriever"]["documents"]]

question = "What medication was prescribed at discharge?"
context = retrieve_context(question)

prompt = f"""
Use only this context:
{context}

Question: {question}
Answer:
"""

print(prompt)

In a real agent stack, pass context into your LLM node or tool-calling layer. Keep patient identifiers out of prompts unless you have a clear compliance boundary and audit trail.

Testing the Integration

Run a simple end-to-end check: write docs, query them back, and confirm retrieval returns the expected clinical note.

test_query = "What was the HbA1c result?"
result = pipe.run({
    "query_embedder": {"text": test_query}
})

hits = result["retriever"]["documents"]
print("Top hit:", hits[0].content)
print("Metadata:", hits[0].meta)

Expected output:

Top hit: HbA1c measured at 8.4%. Recommend diabetes management review.
Metadata: {'doc_type': 'lab_note', 'department': 'pathology', 'patient_hash': 'p_10492', 'source': 'lab_system'}

If that works, your indexing path and retrieval path are both wired correctly.

Real-World Use Cases

• •Clinical QA assistant that answers questions from discharge notes, pathology reports, and care plans with traceable citations.
• •Prior authorization agent that searches policy docs plus patient history summaries to draft evidence-backed submissions.
• •Care navigation bot that retrieves follow-up instructions, medication changes, and referral details from EHR exports.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.