How to Build a policy Q&A Agent Using CrewAI in TypeScript for banking

A policy Q&A agent for banking answers staff or customer-facing questions from approved policy documents, then returns a grounded response with citations and escalation when the policy is unclear. It matters because banking teams need fast answers without drifting into non-compliant advice, and every response has to be auditable, consistent, and constrained to the right jurisdiction.

Architecture

• •

  Policy source layer

  • •Ingests approved PDFs, markdown, or internal knowledge base exports.
  • •Keeps document versioning so you can trace which policy was used for an answer.
• •

  Retrieval layer

  • •Finds the most relevant policy chunks before the model answers.
  • •Uses embeddings or search over a controlled corpus, not open web search.
• •

  CrewAI orchestration layer

  • •Uses a Crew with one or more Agent instances to coordinate retrieval, answer drafting, and compliance checking.
  • •Keeps the logic explicit instead of hiding it inside one prompt.
• •

  Compliance guardrail layer

  • •Rejects requests that ask for legal advice, exceptions, or unsupported product guidance.
  • •Forces citation-backed answers and escalation when confidence is low.
• •

  Audit and logging layer

  • •Stores question, retrieved sources, final answer, model version, and timestamp.
  • •Supports internal review and regulator-facing traceability.

Implementation

1) Install dependencies and define your policy corpus

For TypeScript, keep the setup simple: CrewAI for orchestration, plus whatever retrieval store you use. In practice I recommend a local vector index for development and a controlled managed store in production with residency pinned to your banking region.

npm install @crewai/crewai openai dotenv
npm install -D typescript tsx @types/node

Create a minimal policy loader around approved content only:

// src/policies.ts
export type PolicyDoc = {
  id: string;
  title: string;
  jurisdiction: string;
  version: string;
  content: string;
};

export const policies: PolicyDoc[] = [
  {
    id: "mortgage-ltv-uk-2025",
    title: "Mortgage Lending Policy",
    jurisdiction: "UK",
    version: "2025.01",
    content:
      "Maximum LTV for standard residential mortgages is 90%. Exceptions require credit committee approval. Customer affordability checks must follow FCA guidance.",
  },
  {
    id: "cards-disputes-eu-2025",
    title: "Card Disputes Policy",
    jurisdiction: "EU",
    version: "2025.02",
    content:
      "Chargeback disputes must be logged within scheme time limits. Sensitive customer data must not be copied into free-text notes.",
  },
];

2) Build a retrieval tool that only searches approved policies

The agent should never answer from memory alone. Use a tool that returns matched policy snippets with metadata so the final response can cite them.

// src/tools/policySearch.ts
import { Tool } from "@crewai/crewAI";
import { policies } from "../policies";

export class PolicySearchTool extends Tool {
  name = "policy_search";
  description = "Search approved banking policies by keyword and jurisdiction.";

  async execute(input: string): Promise<string> {
    const query = input.toLowerCase();

    const matches = policies.filter((p) => {
      const haystack = `${p.title} ${p.jurisdiction} ${p.content}`.toLowerCase();
      return haystack.includes(query) || query.split(" ").some((term) => haystack.includes(term));
    });

    if (matches.length === 0) {
      return JSON.stringify({
        found: false,
        message: "No matching approved policy found.",
      });
    }

    return JSON.stringify({
      found: true,
      results: matches.map((p) => ({
        id: p.id,
        title: p.title,
        jurisdiction: p.jurisdiction,
        version: p.version,
        excerpt: p.content,
      })),
    });
  }
}

3) Define the agent and crew with strict instructions

This is the core pattern. One agent retrieves and answers; another can review for compliance if you want a second pass. The important part is that the system prompt forces grounding, citations, and escalation.

// src/agent.ts
import { Agent, Crew, Task } from "@crewai/crewAI";
import { PolicySearchTool } from "./tools/policySearch";

const policyAgent = new Agent({
  role: "Banking Policy Q&A Analyst",
  goal:
    "Answer banking policy questions using only approved internal policies and provide citations.",
  backstory:
    "You work in a regulated bank. You must be precise, conservative, and compliant.",
  tools: [new PolicySearchTool()],
});

const answerTask = new Task({
  description:
    [
      "Answer the user's question using only retrieved approved policies.",
      "If no relevant policy exists, say you cannot confirm and escalate to compliance.",
      "Include cited policy IDs and versions in the response.",
      "Do not provide legal advice or invent exceptions.",
      `Question: {question}`,
    ].join("\n"),
  expectedOutput:
    "A concise banking-safe answer with citations or an escalation message.",
});

export async function answerPolicyQuestion(question: string) {
  const crew = new Crew({
    agents: [policyAgent],
    tasks: [answerTask],
    verbose: true,
  });

  const result = await crew.kickoff({ question });
  return result;
}

4) Add an API boundary with audit logging

In banking, the agent is not your product; it is a controlled service behind an API. Log inputs and outputs with redaction rules so you can prove what was asked and what source was used without storing unnecessary personal data.

// src/server.ts
import express from "express";
import dotenv from "dotenv";
import { answerPolicyQuestion } from "./agent";

dotenv.config();

const app = express();
app.use(express.json());

app.post("/policy-qa", async (req, res) => {
  const { question } = req.body;

  if (!question || typeof question !== "string") {
    return res.status(400).json({ error: "question is required" });
  }

   // Redact obvious PII before logging in production.
   console.log("[policy-qa] question received");

   const result = await answerPolicyQuestion(question);

   console.log("[policy-qa] answer generated");

   res.json({
     answer: result,
   });
});

app.listen(3000, () => {
   console.log("Policy Q&A service listening on port 3000");
});

Production Considerations

• •

  Deployment

  • •Run inside your bank’s approved cloud region or on-prem environment.
  • •Pin model endpoints to jurisdictions that satisfy data residency requirements.
• •

  Monitoring

  • •Track retrieval hit rate, escalation rate, refusal rate, and citation coverage.
  • •Alert when answers are generated without matching policy documents.
• •

  Guardrails

  • •Block requests involving legal interpretation, customer-specific decisions, or exceptions outside policy.
  • •Require explicit escalation paths for ambiguous queries like “Can we approve this anyway?”
• •

  Auditability

  • •Store question hash, retrieved policy IDs, model version, timestamp, and final response.
  • •Keep immutable logs for compliance review and incident investigation.

Common Pitfalls

1. •

   Letting the model answer without retrieval

   • •This turns the agent into a generic chatbot.
   • •Fix it by forcing every task through policy_search before any response is drafted.
2. •

   Mixing jurisdictions in one response

   • •A UK mortgage rule should not be applied to an EU branch by accident.
   • •Fix it by passing jurisdiction as an explicit filter in your retrieval tool and task context.
3. •

   Logging raw customer data

   • •Banking teams often discover this too late during audit.
   • •Fix it by redacting names, account numbers, addresses, and free-text PII before persistence.
4. •

   Treating “no match found” as an answer

   • •If the corpus does not contain the rule, the agent should escalate.
   • •Fix it by making “cannot confirm from approved policy” a valid final output path.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.