How to Integrate Haystack for lending with Elasticsearch for multi-agent systems

Combining Haystack for lending with Elasticsearch gives you a clean pattern for multi-agent systems that need both structured retrieval and domain-specific loan workflows. In practice, this lets one agent pull policy, borrower, and product data from Elasticsearch while another agent uses Haystack’s lending components to reason over eligibility, document completeness, and next-best actions.

Prerequisites

• •Python 3.10+
• •An Elasticsearch cluster running locally or in your environment
• •Access to Haystack for lending packages and their configured lending pipeline components
• •pip installed
• •API keys or credentials for any upstream systems your lending agents query
• •A working understanding of:
  • •Haystack pipelines
  • •Elasticsearch indices and mappings
  • •Agent orchestration patterns

Install the core dependencies:

pip install haystack-ai elasticsearch

If your Haystack for lending distribution is packaged separately in your environment, install that too:

pip install haystack-lending

Integration Steps

1. •Connect to Elasticsearch and create a lending index

Start by storing borrower profiles, loan products, policy rules, and document metadata in Elasticsearch. Use a dedicated index so agents can retrieve only lending-relevant records.

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")

index_name = "lending_knowledge"

mapping = {
    "mappings": {
        "properties": {
            "doc_type": {"type": "keyword"},
            "title": {"type": "text"},
            "content": {"type": "text"},
            "borrower_id": {"type": "keyword"},
            "product_code": {"type": "keyword"},
            "updated_at": {"type": "date"}
        }
    }
}

if not es.indices.exists(index=index_name):
    es.indices.create(index=index_name, **mapping)

es.index(
    index=index_name,
    id="rule-001",
    document={
        "doc_type": "policy",
        "title": "Debt-to-income threshold",
        "content": "For unsecured personal loans, DTI must be below 45%.",
        "product_code": "PERSONAL_LOAN",
        "updated_at": "2026-04-21"
    }
)

es.index(
    index=index_name,
    id="doc-001",
    document={
        "doc_type": "checklist",
        "title": "Mortgage submission checklist",
        "content": "Require payslips, bank statements, proof of address, and ID.",
        "product_code": "MORTGAGE",
        "updated_at": "2026-04-21"
    }
)

2. •Build an Elasticsearch retrieval layer for your agents

In multi-agent systems, don’t let every agent query Elasticsearch directly with ad hoc filters. Wrap retrieval in a single function so the lending agent can ask for context consistently.

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")
INDEX = "lending_knowledge"

def search_lending_context(query: str, product_code: str | None = None):
    must_clauses = [
        {"multi_match": {"query": query, "fields": ["title^2", "content"]}}
    ]

    if product_code:
        must_clauses.append({"term": {"product_code": product_code}})

    response = es.search(
        index=INDEX,
        size=5,
        query={"bool": {"must": must_clauses}}
    )

    return [
        {
            "_id": hit["_id"],
            "_score": hit["_score"],
            **hit["_source"]
        }
        for hit in response["hits"]["hits"]
    ]

results = search_lending_context("required documents for mortgage", product_code="MORTGAGE")
print(results)

3. •Wire the retrieval into a Haystack pipeline

Use Haystack as the orchestration layer for the lending workflow. A common pattern is: retrieve context from Elasticsearch, then feed it into a prompt builder or generator node that produces a decision summary for another agent.

from haystack import Document, Pipeline
from haystack.components.builders import PromptBuilder

template = """
You are a lending operations assistant.

Use the retrieved policy and checklist context to answer:
{{question}}

Context:
{% for doc in documents %}
- {{ doc.meta.title }}: {{ doc.content }}
{% endfor %}

Return:
1. Eligibility notes
2. Missing documents
3. Recommended next action
"""

prompt_builder = PromptBuilder(template=template)

def to_haystack_documents(search_results):
    return [
        Document(content=item["content"], meta={
            "title": item["title"],
            "doc_type": item["doc_type"],
            "product_code": item["product_code"]
        })
        for item in search_results
    ]

pipeline = Pipeline()
pipeline.add_component("prompt_builder", prompt_builder)

query = "What documents are missing for a mortgage application?"
docs = to_haystack_documents(search_lending_context(query, product_code="MORTGAGE"))

result = prompt_builder.run(
    question=query,
    documents=docs
)

print(result["prompt"])

4. •Add an agent-facing tool wrapper

For multi-agent systems, expose the retrieval-and-reasoning flow as a callable tool. One agent can handle borrower conversation while another uses this tool to fetch lending evidence.

from typing import Any

def lending_knowledge_tool(question: str, product_code: str) -> dict[str, Any]:
    docs = search_lending_context(question, product_code=product_code)
    
    haystack_docs = [
        Document(content=d["content"], meta={
            "title": d["title"],
            "doc_type": d["doc_type"],
            "product_code": d["product_code"]
        })
        for d in docs
    ]

    prompt_result = prompt_builder.run(
        question=question,
        documents=haystack_docs
    )

    return {
        "retrieved_documents": docs,
        "prompt_input": prompt_result["prompt"]
    }

tool_output = lending_knowledge_tool(
    question="Can this applicant proceed with underwriting?",
    product_code="PERSONAL_LOAN"
)

print(tool_output["prompt_input"])

5. •Persist agent decisions back into Elasticsearch

This closes the loop. Store eligibility decisions, missing-doc flags, and handoff notes so downstream agents can pick up state without recomputing everything.

decision_index = "lending_agent_decisions"

if not es.indices.exists(index=decision_index):
    es.indices.create(index=decision_index)

decision_doc = {
    "borrower_id": "B12345",
    "product_code": "MORTGAGE",
    "decision_status": “needs_review”,
    “reason”: “Missing bank statements from last 90 days”,
    “created_at”: “2026-04-21”
}

es.index(
    index=decision_index,
    id="decision-B12345",
    document=decision_doc
)

Testing the Integration

Run an end-to-end check by indexing a policy record, retrieving it through Elasticsearch, and passing it into the Haystack prompt builder.

query = “What is the DTI threshold for unsecured personal loans?”
results = search_lending_context(query, product_code="PERSONAL_LOAN")

assert len(results) > 0

docs = to_haystack_documents(results)
output = prompt_builder.run(question=query, documents=docs)

print("Retrieved:", results[0]["title"])
print("Prompt preview:")
print(output["prompt"][:300])

Expected output:

Retrieved: Debt-to-income threshold
Prompt preview:
You are a lending operations assistant.

Use the retrieved policy and checklist context to answer:
What is the DTI threshold for unsecured personal loans?
...

Real-World Use Cases

• •

  Loan pre-screening agents

  • •One agent pulls borrower facts from operational data.
  • •Another retrieves policy constraints from Elasticsearch.
  • •Haystack generates a structured recommendation for pre-approval or manual review.
• •

  Document completeness checks

  • •Store required-doc rules by loan type in Elasticsearch.
  • •Let a Haystack-backed agent compare submitted docs against those rules.
  • •Return missing items before underwriting starts.
• •

  Policy-aware support agents

  • •Customer-facing agents answer questions using indexed lending policies.
  • •Internal agents use the same index to explain exceptions and escalation paths.
  • •You keep answers consistent across channels and teams.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.