How to Build a policy Q&A Agent Using AutoGen in TypeScript for insurance

A policy Q&A agent answers customer or internal questions against insurance policy documents, endorsements, and product guides. The value is simple: it reduces call-center load, speeds up claims and underwriting support, and gives consistent answers that stay grounded in the actual policy wording.

Architecture

• •

  User interface

  • •Chat UI, internal agent console, or API endpoint.
  • •Keep it thin; all policy logic should live in the agent layer.
• •

  Policy retrieval layer

  • •Vector search over policy PDFs, riders, exclusions, and FAQs.
  • •Use chunking with metadata like policyId, jurisdiction, effectiveDate, and documentType.
• •

  AutoGen assistant agent

  • •The main LLM-driven responder.
  • •Responsible for asking clarifying questions and producing a final answer with citations.
• •

  Tooling layer

  • •Functions for retrieving policy passages, checking eligibility rules, and logging audit events.
  • •Avoid letting the model “guess” coverage without retrieval.
• •

  Compliance and guardrails

  • •Redaction for PII, refusal rules for legal advice, and jurisdiction checks.
  • •Every answer should be traceable to source text.
• •

  Audit store

  • •Persist user question, retrieved chunks, final answer, model version, and timestamps.
  • •This matters when a regulator or claims reviewer asks why the agent answered a certain way.

Implementation

1) Install dependencies and define your policy search tool

For TypeScript, use AutoGen’s official package plus a real retrieval backend. The pattern below assumes you already indexed policy text into your own search service.

npm install @autogen-ai/autogen openai zod

import { AssistantAgent } from "@autogen-ai/autogen";
import { z } from "zod";

type PolicyChunk = {
  text: string;
  source: string;
  page: number;
  policyId: string;
};

async function searchPolicyCorpus(query: string): Promise<PolicyChunk[]> {
  // Replace with Pinecone / Azure AI Search / pgvector / Elastic
  return [
    {
      text: "Coverage applies only when the loss occurs during the policy period.",
      source: "homeowners-policy.pdf",
      page: 12,
      policyId: "HO-2024-001",
    },
  ];
}

2) Create an AutoGen assistant with strict instructions

This agent should not freewheel. It should retrieve first, answer second, and cite sources every time. The system message needs to force that behavior.

const assistant = new AssistantAgent({
  name: "policy_qa_agent",
  modelClient: {
    model: "gpt-4o-mini",
    apiKey: process.env.OPENAI_API_KEY!,
    temperature: 0,
  },
  systemMessage: `
You are an insurance policy Q&A agent.
Rules:
1. Answer only from retrieved policy text or clearly say you cannot determine coverage.
2. Always cite source document names and page numbers.
3. If jurisdiction or effective date is missing, ask a clarifying question.
4. Never provide legal advice.
5. Do not expose personal data in the response.
`,
});

3) Wrap retrieval in a tool-like function and build the response flow

AutoGen’s core pattern is to keep orchestration in your app code and let the agent handle language generation. In production insurance workflows, that means you fetch evidence first, then pass that evidence into the agent as context.

const QuestionSchema = z.object({
  question: z.string().min(5),
});

export async function answerPolicyQuestion(input: unknown) {
  const { question } = QuestionSchema.parse(input);

  const chunks = await searchPolicyCorpus(question);

  const context = chunks
    .map(
      (c) =>
        `SOURCE: ${c.source} | PAGE: ${c.page} | POLICY: ${c.policyId}\n${c.text}`
    )
    .join("\n\n");

  const prompt = `
User question:
${question}

Retrieved policy excerpts:
${context}

Instructions:
- Answer using only the excerpts above.
- If the excerpts do not support an answer, say so explicitly.
- Include bullet-point citations in the format [source.pdf p.X].
`;

  const result = await assistant.run([{ role: "user", content: prompt }]);

  return {
    answer: result.messages.at(-1)?.content ?? "",
    sources: chunks.map((c) => ({
      source: c.source,
      page: c.page,
      policyId: c.policyId,
    })),
  };
}

4) Add an audit log before returning the answer

Insurance teams need traceability. Store the user question, retrieved sources, and final output so compliance can review it later.

async function writeAuditEvent(event: {
  question: string;
  answer: string;
  sources: Array<{ source: string; page: number; policyId: string }>;
}) {
  console.log(JSON.stringify({ ...event, ts: new Date().toISOString() }));
}

async function handleRequest(questionText: string) {
  const response = await answerPolicyQuestion({ question: questionText });

  await writeAuditEvent({
    question: questionText,
    answer: response.answer,
    sources: response.sources,
  });

  return response;
}

Production Considerations

• •

  Data residency

  • •Keep document storage and vector indexes in-region if your policies require it.
  • •For multi-country insurers, route requests by jurisdiction so data does not cross borders unnecessarily.
• •

  Compliance controls

  • •Block answers when required fields are missing:
    • •jurisdiction
    • •policy effective date
    • •product line
    • •insured type
  • •Add a refusal path for legal interpretation or claim denial decisions.
• •

  Monitoring

  • •Track retrieval hit rate, unanswered questions, citation coverage, and escalation rate.
  • •Alert when the agent answers without enough supporting text or starts citing stale documents.
• •

  Guardrails

  • •Redact PII before sending prompts to the model.
  • •Enforce “answer only from sources” at both prompt level and application level.
  • •Log every model version change because insurance answers must be reproducible.

Common Pitfalls

• •

  Letting the model answer without retrieval

  This creates confident nonsense around exclusions, waiting periods, or sublimits. Always fetch relevant passages first and make citation mandatory.

• •

  Ignoring jurisdiction and effective date

  A homeowners rule in one state may not apply in another. Pass these as structured inputs or ask follow-up questions before answering coverage questions.

• •

  Skipping audit logging

  If a customer disputes an answer or compliance reviews it later, you need the exact prompt context and source documents used. Log inputs, outputs, timestamps, model version, and document IDs.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.