How to Build a customer support Agent Using LlamaIndex in Python for pension funds

A customer support agent for a pension fund answers member questions from policy documents, FAQs, contribution rules, retirement options, and service procedures. It matters because most support tickets are repetitive, regulated, and time-sensitive: if the agent gives the wrong answer on eligibility, withdrawals, or tax treatment, you create compliance risk and real financial harm.

Architecture

• •

  Document ingestion layer

  • •Pulls PDFs, HTML pages, policy docs, benefit guides, and internal SOPs into a normalized corpus.
  • •For pension funds, this should include versioned documents with effective dates.
• •

  Indexing layer

  • •Uses VectorStoreIndex from LlamaIndex to make policy content retrievable by semantic search.
  • •Add metadata like document_type, jurisdiction, effective_date, and confidentiality_level.
• •

  Retriever

  • •Uses index.as_retriever() to fetch the most relevant chunks for each question.
  • •In pension support, retrieval should prefer the latest approved policy and jurisdiction-specific rules.
• •

  Response engine

  • •Uses RetrieverQueryEngine or index.as_query_engine() to generate grounded answers from retrieved context.
  • •The response should cite source chunks and avoid unsupported claims.
• •

  Guardrails and escalation

  • •Detects out-of-scope questions like legal advice, complaints, or account-specific actions.
  • •Escalates to a human agent when confidence is low or when the request touches protected data.
• •

  Audit and observability

  • •Logs prompts, retrieved sources, final answers, and user identifiers for audit trails.
  • •This is non-negotiable in pension operations where every answer may be reviewed later.

Implementation

1. •Install dependencies and load your pension documents

Use LlamaIndex’s file readers to ingest policy PDFs or text files. In production you would likely wrap this with a document pipeline that also extracts metadata from your DMS.

from llama_index.core import SimpleDirectoryReader

# Example directory structure:
# data/
#   member_handbook.pdf
#   contribution_policy.pdf
#   retirement_options.txt

documents = SimpleDirectoryReader(
    input_dir="./data",
    recursive=True
).load_data()

print(f"Loaded {len(documents)} documents")

2. •Build a vector index with metadata-aware documents

For pension funds, metadata matters. You want retrieval to prefer current policy versions and the correct jurisdiction.

from llama_index.core import VectorStoreIndex

for doc in documents:
    doc.metadata = {
        "source_system": "pension_knowledge_base",
        "jurisdiction": "UK",
        "content_class": "member_support"
    }

index = VectorStoreIndex.from_documents(documents)

3. •Create a query engine that answers only from indexed content

This is the core pattern. The query engine retrieves relevant chunks and synthesizes an answer grounded in those chunks.

from llama_index.core import Settings
from llama_index.llms.openai import OpenAI

Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)
query_engine = index.as_query_engine(
    similarity_top_k=4,
    response_mode="compact"
)

question = (
    "Can a member transfer their pension if they are under retirement age?"
)
response = query_engine.query(question)

print(response)

If you want stricter control over retrieval and citations, use RetrieverQueryEngine explicitly:

from llama_index.core.query_engine import RetrieverQueryEngine

retriever = index.as_retriever(similarity_top_k=4)
query_engine = RetrieverQueryEngine.from_args(retriever)

response = query_engine.query(
    "What happens if a member stops contributing for six months?"
)

print(response)

4. •Add a simple escalation rule for regulated cases

Pension support should not answer everything automatically. If the question implies legal interpretation, personal account action, or complaint handling, route it to a human queue.

def should_escalate(question: str) -> bool:
    keywords = [
        "complaint",
        "appeal",
        "legal",
        "lawsuit",
        "my account",
        "withdraw my money",
        "beneficiary change"
    ]
    q = question.lower()
    return any(k in q for k in keywords)

user_question = "I want to withdraw my pension early because I lost my job."
if should_escalate(user_question):
    print("Escalate to human support.")
else:
    print(query_engine.query(user_question))

Production Considerations

• •

  Data residency

  • •Keep document storage, vector indexes, and logs in the region required by your regulator or internal policy.
  • •For cross-border pension operations, separate indexes by jurisdiction instead of mixing everything into one global store.
• •

  Auditability

  • •Persist the user question, retrieved node IDs, source document names, model version, and final response.
  • •When compliance asks “why did the bot say that?”, you need traceability down to the chunk level.
• •

  Guardrails

  • •Block account-specific actions unless the agent is integrated with authenticated member systems.
  • •Add an escalation path for complaints, tax advice, transfers between schemes, death benefits, and early-access requests.
• •

  Monitoring

  • •Track retrieval quality: top-k hit rate, fallback rate, escalation rate, and hallucination reports.
  • •Watch for drift when policies change; stale embeddings can keep surfacing old rules after updates.

Common Pitfalls

1. •

   Using stale policy documents

   • •Pension rules change often. If you don’t version documents and reindex after approvals, the agent will answer with outdated guidance.
   • •Fix it by attaching effective_date metadata and filtering retrieval to approved/current content only.
2. •

   Letting the model answer beyond retrieved evidence

   • •A generic chat model will confidently invent details about tax treatment or withdrawal eligibility.
   • •Fix it by grounding responses in retrieved nodes only and escalating low-confidence or sensitive queries.
3. •

   Ignoring compliance boundaries

   • •Support agents for pension funds are not general financial advisors.
   • •Fix it by blocking legal/tax advice prompts, requiring human review for sensitive cases, and logging every interaction for audit.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.