How to Build a policy Q&A Agent Using CrewAI in Python for fintech

A policy Q&A agent for fintech answers questions about internal policies, product rules, compliance requirements, and operational procedures. The point is not just convenience; it reduces time spent searching docs, keeps responses consistent, and gives you an auditable layer between employees or customers and sensitive policy content.

Architecture

• •

  Policy corpus ingestion

  • •Load policy PDFs, markdown docs, and runbooks into a searchable store.
  • •For fintech, keep source-of-truth documents versioned and tied to approval dates.
• •

  Retriever tool

  • •Expose a tool that fetches the most relevant policy passages for a question.
  • •This should support citation-ready snippets, not just raw text.
• •

  Policy analyst agent

  • •Uses CrewAI Agent to interpret the question and draft an answer from retrieved context.
  • •Its job is synthesis, not free-form guessing.
• •

  Compliance reviewer agent

  • •A second Agent checks the draft against guardrails: no legal advice, no unsupported claims, no leakage of restricted data.
  • •In regulated environments, this step is not optional.
• •

  Execution workflow

  • •A CrewAI Crew coordinates the agents through Tasks.
  • •The workflow should force retrieval first, then answer generation, then compliance review.
• •

  Audit logging layer

  • •Store question, retrieved sources, final answer, model used, and timestamp.
  • •Fintech teams need traceability for internal audits and incident reviews.

Implementation

1) Install dependencies and prepare your policy index

Use CrewAI plus a vector store or document loader of your choice. The example below assumes you already have policy text available locally and want a minimal production pattern around retrieval.

pip install crewai crewai-tools langchain-openai chromadb

import os
from pathlib import Path

from crewai import Agent, Task, Crew, Process
from crewai_tools import tool

POLICY_DIR = Path("./policies")

def load_policy_files():
    docs = []
    for file in POLICY_DIR.glob("**/*.md"):
        docs.append({
            "source": str(file),
            "text": file.read_text(encoding="utf-8")
        })
    return docs

POLICIES = load_policy_files()

2) Create a retrieval tool with explicit citations

The agent should answer from approved content only. This example uses a simple keyword matcher so the pattern is clear; swap it with your vector DB retriever in production.

@tool("search_policy_corpus")
def search_policy_corpus(query: str) -> str:
    """
    Search approved policy documents and return relevant excerpts with sources.
    """
    query_terms = set(query.lower().split())
    matches = []

    for doc in POLICIES:
        text = doc["text"].lower()
        score = sum(1 for term in query_terms if term in text)
        if score > 0:
            excerpt = doc["text"][:1200]
            matches.append(f"SOURCE: {doc['source']}\nEXCERPT:\n{excerpt}")

    if not matches:
        return "No relevant policy excerpts found."

    return "\n\n---\n\n".join(matches[:3])

3) Define the agents and tasks

Use one agent to answer and another to verify. Keep both constrained to the same source context so you can audit what influenced the response.

policy_analyst = Agent(
    role="Policy Analyst",
    goal="Answer fintech policy questions using only approved internal sources.",
    backstory=(
        "You are an internal policy specialist at a fintech company. "
        "You never invent policy details and always cite sources."
    ),
    tools=[search_policy_corpus],
    verbose=True,
)

compliance_reviewer = Agent(
    role="Compliance Reviewer",
    goal="Review answers for compliance risk, unsupported claims, and missing citations.",
    backstory=(
        "You enforce fintech controls: no legal advice, no speculation, "
        "no disclosure of restricted information."
    ),
    verbose=True,
)

answer_task = Task(
    description=(
        "Answer the user's policy question using only retrieved policy excerpts. "
        "If the source material does not contain enough information, say so clearly."
    ),
    expected_output="A concise answer with citations to source documents.",
    agent=policy_analyst,
)

review_task = Task(
    description=(
        "Review the drafted answer for compliance issues. "
        "Flag any unsupported statements or missing citations. "
        "Return either APPROVED or REJECTED with reasons."
    ),
    expected_output="Compliance review result with specific findings.",
    agent=compliance_reviewer,
)

4) Run the Crew and capture output for audit

This is the core execution pattern. In production you would wrap this behind an API endpoint and persist every request/response pair to your audit store.

def run_policy_qna(question: str):
    crew = Crew(
        agents=[policy_analyst, compliance_reviewer],
        tasks=[answer_task, review_task],
        process=Process.sequential,
        verbose=True,
    )

    result = crew.kickoff(inputs={"question": question})

    return {
        "question": question,
        "answer": str(result),
    }

if __name__ == "__main__":
    response = run_policy_qna(
        "Can customer support disclose account closure reasons over email?"
    )
    print(response["answer"])

A better production pattern is to pass the retrieved context explicitly into the task input so you can store exactly what was used. That makes audits easier when someone asks why a response was generated.

Production Considerations

• •

  Guardrails for regulated language

  • •Block answers that sound like legal advice or guarantee regulatory outcomes.
  • •Require phrases like “per policy” or “according to document X” when making factual claims.
• •

  Auditability

  • •Log question text, retrieved excerpts, final response, model name, prompt version, and user identity.
  • •For fintech audits, keep immutable logs with retention aligned to your control framework.
• •

  Data residency

  • •If policies contain region-specific customer or operational data, keep retrieval and inference inside approved regions.
  • •Don’t send sensitive internal policies to external endpoints without explicit approval from security and legal.
• •

  Monitoring

  • •Track unsupported-answer rate, retrieval hit rate, escalation rate, and review rejections.
  • •Alert when the agent starts answering too confidently from weak evidence.

Common Pitfalls

1. •

   Letting the agent answer without retrieval

   • •This turns your system into a general chatbot with policy-flavored hallucinations.
   • •Fix it by forcing every task to consume retrieved excerpts before generating an answer.
2. •

   Skipping compliance review

   • •One-agent setups are faster to build but weak for fintech controls.
   • •Fix it by adding a reviewer Agent that rejects unsupported statements before anything reaches users.
3. •

   Not versioning policy sources

   • •If policies change weekly and your index does not track versions, you cannot explain old answers.
   • •Fix it by storing document version IDs and timestamps alongside every retrieved excerpt and final response.
4. •

   Ignoring regional restrictions

   • •A Q&A agent that moves sensitive policies across regions can create residency violations.
   • •Fix it by pinning storage and model execution to approved infrastructure per jurisdiction.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.