How to Build a policy Q&A Agent Using CrewAI in Python for retail banking

A policy Q&A agent for retail banking answers staff or customer-facing questions from approved policy documents, procedure manuals, and product disclosures. It matters because banking teams need fast, consistent answers without drifting into non-compliant advice, exposing sensitive data, or giving outdated guidance.

Architecture

• •

  Policy ingestion layer

  • •Loads approved PDFs, DOCX files, and internal knowledge base articles.
  • •Normalizes content into chunks with metadata like document version, jurisdiction, product line, and effective date.
• •

  Retriever tool

  • •Searches only approved policy sources.
  • •Returns the most relevant excerpts with citations so the agent can ground every answer.
• •

  Q&A agent

  • •Uses CrewAI Agent with a strict role like “Retail Banking Policy Analyst.”
  • •Answers only from retrieved context and refuses unsupported claims.
• •

  Compliance reviewer

  • •A second agent that checks the draft answer for policy drift, prohibited advice, missing disclaimers, and regulatory sensitivity.
• •

  Crew orchestration

  • •Uses CrewAI Task and Crew to coordinate retrieval, answer generation, and review in sequence.
  • •Keeps the flow auditable for internal controls.
• •

  Audit logging

  • •Stores question text, retrieved sources, final answer, model version, timestamps, and reviewer output.
  • •Supports post-incident review and regulatory traceability.

Implementation

1. •

   Install CrewAI and define your policy corpus

   Use a small set of approved documents first. In retail banking, your source list should be curated by compliance or product owners; do not let the agent crawl random shared drives.

   pip install crewai crewai-tools python-dotenv
   

   import os
   from dotenv import load_dotenv
   
   load_dotenv()
   os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "")
   

2. •

   Create tools for policy lookup and citation

   CrewAI works well when the agent is constrained to explicit tools. For production, swap this in-memory example with a vector store or search index backed by approved documents.

   from typing import List
   from crewai.tools import BaseTool
   
   POLICY_STORE = [
       {
           "id": "deposit_policy_001",
           "title": "Retail Deposit Account Policy",
           "jurisdiction": "US",
           "effective_date": "2025-01-01",
           "text": "Overdraft fees may be waived once per calendar year for eligible customers."
       },
       {
           "id": "cards_policy_014",
           "title": "Debit Card Dispute Policy",
           "jurisdiction": "US",
           "effective_date": "2025-02-15",
           "text": "Unauthorized debit card transactions must be reported within 60 days of statement date."
       }
   ]
   
   class PolicySearchTool(BaseTool):
       name: str = "policy_search"
       description: str = "Search approved retail banking policies by keyword."
   
       def _run(self, query: str) -> str:
           matches: List[str] = []
           q = query.lower()
           for doc in POLICY_STORE:
               if any(token in doc["text"].lower() or token in doc["title"].lower() for token in q.split()):
                   matches.append(
                       f"[{doc['id']}] {doc['title']} | {doc['jurisdiction']} | "
                       f"{doc['effective_date']} | {doc['text']}"
                   )
           return "\n".join(matches) if matches else "No approved policy match found."
   

3. •

   Define the answering and compliance agents

   The main agent should be conservative. The reviewer should look for unsupported claims and anything that sounds like regulated advice rather than policy explanation.

   from crewai import Agent
   
   policy_search_tool = PolicySearchTool()
   
   answer_agent = Agent(
       role="Retail Banking Policy Analyst",
       goal="Answer questions using only approved banking policies and cite sources.",
       backstory=(
           "You support retail banking operations. You never guess. "
           "If the source does not support the answer, you say you cannot confirm it."
       ),
       tools=[policy_search_tool],
       verbose=True,
       allow_delegation=False,
   )
   
   compliance_agent = Agent(
       role="Banking Compliance Reviewer",
       goal="Review answers for policy accuracy, compliance risk, and missing citations.",
       backstory=(
           "You check answers against retail banking standards. "
           "You flag unsupported statements, ambiguous wording, and missing disclaimers."
       ),
       verbose=True,
       allow_delegation=False,
   )
   

4. •

   Orchestrate the flow with tasks and a crew

   This is the pattern you want: retrieve first, draft second, review third. The final answer should be returned only after review passes.

   from crewai import Task, Crew
   
   question = (
       "What is the timeframe for reporting an unauthorized debit card transaction?"
   )
   
   retrieval_task = Task(
       description=(
           f"Find all approved policies relevant to this question: {question}. "
           f"Return exact excerpts with document IDs."
       ),
       expected_output="Relevant policy excerpts with citations.",
       agent=answer_agent,
   )
   
   draft_task = Task(
       description=(
           f"Using only the retrieved policy excerpts, draft a concise answer to: {question}. "
           f"Include citations inline using document IDs."
       ),
       expected_output="A compliant draft answer with citations.",
       agent=answer_agent,
       context=[retrieval_task],
   )
   
   review_task = Task(
       description=(
           "Review the draft answer for compliance risk. "
           "Reject any unsupported claim or missing citation. "
           "Return either APPROVED or REVISE with reasons."
       ),
       expected_output="APPROVED or REVISE plus rationale.",
       agent=compliance_agent,
       context=[draft_task],
   )
   
   crew = Crew(
       agents=[answer_agent, compliance_agent],
       tasks=[retrieval_task, draft_task, review_task],
       verbose=True,
   )
   
   result = crew.kickoff()
   print(result)
   

Production Considerations

• •

  Deploy in-region

  • •Retail banking often has data residency constraints.
  • •Keep model calls, logs, embeddings storage, and vector indexes inside approved regions such as US-only or EU-only deployments.
• •

  Log every decision path

  • •Store user question, retrieved passages, final response, reviewer outcome, model name/version, and timestamps.
  • •Make audit logs immutable where possible so compliance can reconstruct how an answer was produced.
• •

  Add hard guardrails

  • •Block account-specific advice unless authenticated systems are explicitly integrated.
  • •Refuse questions that require legal interpretation or individualized financial advice; route those to human support.
• •

  Monitor drift and source freshness

  • •Track which document versions are being cited.
  • •Alert when policies expire or when answers are generated from stale content after a product change or regulatory update.

Common Pitfalls

• •

  Letting the agent answer without grounded sources

  • •Bad pattern: asking the LLM to “just explain” policies from memory.
  • •Fix: require retrieval first and reject outputs without citations to approved documents.
• •

  Mixing public LLM access with sensitive customer data

  • •Bad pattern: sending PII or account details into prompts.
  • •Fix: redact identifiers before inference and use authenticated backend services for account-specific lookups.
• •

  Treating compliance as a prompt instead of a control

  • •Bad pattern: one system prompt saying “be compliant.”
  • •Fix: add a separate review task or rule-based validator that can block unsafe responses before they reach users.

If you build this way, CrewAI gives you a clean orchestration layer while your real control points stay where they belong: approved sources, explicit citations, audit logs, and human-reviewed guardrails. That is the difference between a useful banking assistant and an incident waiting to happen.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.