How to Build a policy Q&A Agent Using CrewAI in Python for investment banking

A policy Q&A agent for investment banking answers internal questions about policies, procedures, and controls with traceable citations. It matters because bankers and operations teams need fast answers without opening the door to compliance drift, stale guidance, or unapproved advice.

Architecture

• •

  Policy ingestion layer

  • •Pulls PDFs, SharePoint exports, Confluence pages, or internal policy docs.
  • •Normalizes them into text chunks with metadata like document owner, effective date, jurisdiction, and version.
• •

  Retrieval layer

  • •Uses a vector store or keyword index to fetch the most relevant policy sections.
  • •Must preserve source references so every answer can be audited later.
• •

  CrewAI agent layer

  • •One agent handles policy interpretation.
  • •Another agent validates the answer against compliance constraints.
  • •A third agent can format the final response for bankers in plain language.
• •

  Tooling layer

  • •Search tool for policy lookup.
  • •Citation tool for returning exact passages.
  • •Optional approval/escalation tool when the question touches restricted topics like MNPI, conflicts, gifts, or trade surveillance.
• •

  Guardrail layer

  • •Blocks unsupported advice.
  • •Forces “I don’t know” when retrieval confidence is low.
  • •Requires citations for every substantive claim.
• •

  Audit and observability layer

  • •Logs user question, retrieved sources, final answer, model version, and timestamp.
  • •Needed for internal audit, compliance review, and incident reconstruction.

Implementation

1. •Install CrewAI and define a retrieval tool

Use a real CrewAI BaseTool so the agent can query your policy corpus. In production you would back this with a vector database or search service; here the tool stub shows the actual integration pattern.

from crewai.tools import BaseTool
from pydantic import BaseModel, Field

class PolicySearchInput(BaseModel):
    query: str = Field(..., description="User question about bank policy")

class PolicySearchTool(BaseTool):
    name: str = "policy_search"
    description: str = "Search internal banking policies and return relevant excerpts with citations."
    args_schema = PolicySearchInput

    def _run(self, query: str) -> str:
        # Replace this with vector DB / search backend
        results = [
            {
                "source": "Global Gifts & Entertainment Policy v4.2",
                "section": "3.1 Thresholds",
                "text": "Employees may accept gifts up to $100 per calendar year from a single counterparty.",
            },
            {
                "source": "Global Gifts & Entertainment Policy v4.2",
                "section": "3.4 Escalation",
                "text": "Anything above threshold must be reported to Compliance within one business day.",
            },
        ]
        return "\n".join(
            f"[{r['source']} | {r['section']}] {r['text']}" for r in results
        )

2. •Create specialized agents for answer generation and compliance review

For investment banking, do not use one generic agent. Split interpretation from control validation so you can inspect each step independently.

from crewai import Agent

policy_analyst = Agent(
    role="Policy Analyst",
    goal="Answer internal investment banking policy questions using only retrieved policy text.",
    backstory="You work inside a bank's controls function and must cite sources exactly.",
    tools=[PolicySearchTool()],
    verbose=True,
)

compliance_reviewer = Agent(
    role="Compliance Reviewer",
    goal="Check answers for policy accuracy, missing citations, and prohibited advice.",
    backstory="You validate responses against bank compliance standards before release.",
    verbose=True,
)

3. •Wire the agents into tasks and a crew

Use Task, Crew, and Process.sequential so the output of the first task feeds the second. This is the simplest pattern that still gives you an auditable workflow.

from crewai import Task, Crew, Process

question = "Can I accept a lunch invitation from an M&A client worth $180?"

analysis_task = Task(
    description=(
        "Answer the user's question using only the policy_search tool output. "
        "Include citations in brackets after each factual statement."
    ),
    expected_output="A concise answer with citations and escalation guidance if needed.",
    agent=policy_analyst,
)

review_task = Task(
    description=(
        "Review the drafted answer for compliance risk. "
        "Reject any unsupported claim or missing citation."
    ),
    expected_output="Approved answer or correction notes.",
    agent=compliance_reviewer,
)

crew = Crew(
    agents=[policy_analyst, compliance_reviewer],
    tasks=[analysis_task, review_task],
    process=Process.sequential,
    verbose=True,
)

result = crew.kickoff(inputs={"question": question})
print(result)

4. •Add a production response contract

For banking use cases, force a strict response format so downstream apps can render it safely. The contract should include answer text, cited sources, confidence level, and escalation status.

def format_response(raw_result: str) -> dict:
    return {
        "answer": raw_result,
        "confidence": "medium",
        "escalate_to_compliance": True if "$180" in raw_result else False,
        "sources_required": True,
    }

Production Considerations

• •

  Compliance by design

  • •Make citations mandatory.
  • •Reject responses that mention thresholds or approvals without source text.
  • •Route sensitive categories like MNPI, sanctions, insider trading, gifts/entertainment, personal account dealing, and wall-crossing to compliance review.
• •

  Auditability

  • •Store user prompt, retrieved passages, model output, reviewer output, and final response.
  • •Keep immutable logs with timestamps and document versions.
  • •Internal audit will ask which policy version was used at answer time.
• •

  Data residency

  • •Keep embeddings, logs, and model traffic inside approved regions.
  • •Do not send confidential policies to unmanaged SaaS endpoints.
  • •If your bank has regional restrictions by entity or booking center, enforce them at retrieval time.
• •

  Monitoring

  • •Track retrieval hit rate, citation coverage, escalation rate, and “I don’t know” frequency.
  • •Alert on answers generated from stale documents or low-confidence retrievals.
  • •Review false positives where the system escalates too aggressively; bankers will bypass it if it is noisy.

Common Pitfalls

• •

  Using one agent for everything

  • •Mistake: letting a single agent retrieve policies, interpret them, and approve them.
  • •Fix: split retrieval/analysis from compliance review so failures are isolated and auditable.
• •

  Skipping source grounding

  • •Mistake: allowing free-form answers without exact citations.
  • •Fix: require every factual claim to reference a retrieved excerpt or return “not enough information.”
• •

  Ignoring policy versioning

  • •Mistake: indexing documents without effective dates or ownership metadata.
  • •Fix: attach version fields during ingestion and prefer current approved policies over archived ones.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.