How to Build a policy Q&A Agent Using LangGraph in TypeScript for healthcare

A policy Q&A agent for healthcare answers questions like “Is this procedure covered?”, “What’s the prior auth rule?”, or “Which policy applies to this member?” by routing the user query through retrieval, policy classification, and grounded response generation. It matters because healthcare teams need fast answers, but they also need traceability, compliance, and strict control over what source material the agent can use.

Architecture

• •

  Chat input layer

  • •Accepts clinician, admin, or member-service questions.
  • •Normalizes the query and attaches tenant, region, and user role metadata.
• •

  Policy retriever

  • •Pulls from approved policy documents only.
  • •Uses metadata filters for plan, state, line of business, and effective date.
• •

  Stateful LangGraph workflow

  • •Orchestrates routing between retrieval, answer generation, and fallback paths.
  • •Keeps structured state so every step is auditable.
• •

  Grounded answer generator

  • •Produces an answer only from retrieved policy text.
  • •Returns citations and confidence signals.
• •

  Compliance guardrail layer

  • •Blocks PHI leakage and unsupported claims.
  • •Enforces “answer from policy only” behavior.
• •

  Audit logging

  • •Stores question, sources used, model version, decision path, and final response.
  • •Needed for post-incident review and regulatory audits.

Implementation

1) Define the graph state and typed inputs

You want explicit state. Healthcare workflows fail when developers pass untyped blobs around and lose track of what was retrieved versus what was generated.

import { StateGraph, START, END } from "@langchain/langgraph";
import { ChatOpenAI } from "@langchain/openai";

type PolicySource = {
  id: string;
  title: string;
  excerpt: string;
  url: string;
};

type AgentState = {
  question: string;
  tenantId: string;
  region: string;
  userRole: "member" | "agent" | "clinician";
  retrievedSources: PolicySource[];
  answer?: string;
  needsEscalation?: boolean;
};

const model = new ChatOpenAI({
  model: "gpt-4o-mini",
  temperature: 0,
});

async function classifyNeed(state: AgentState): Promise<Partial<AgentState>> {
  const q = state.question.toLowerCase();
  return {
    needsEscalation:
      q.includes("diagnosis") ||
      q.includes("medical advice") ||
      q.includes("treatment recommendation"),
  };
}

2) Add retrieval with healthcare-specific filters

In healthcare, retrieval is not just semantic search. You must filter by tenant, jurisdiction, plan type, and effective date so the agent does not cite the wrong policy.

async function retrievePolicies(state: AgentState): Promise<Partial<AgentState>> {
  // Replace with your vector store + metadata filter implementation.
  const sources: PolicySource[] = [
    {
      id: "policy-123",
      title: "Prior Authorization Policy",
      excerpt:
        "MRI requires prior authorization for commercial plans in CA when ordered outpatient.",
      url: "https://policies.example.com/policy-123",
    },
    {
      id: "policy-456",
      title: "Coverage Exceptions Policy",
      excerpt:
        "Urgent inpatient imaging may be exempt from prior authorization if clinically justified.",
      url: "https://policies.example.com/policy-456",
    },
  ];

  return { retrievedSources: sources };
}

3) Generate a grounded answer with citations

The key pattern is to force the model to answer only from retrieved text. If there are no relevant sources, it should escalate instead of guessing.

async function generateAnswer(state: AgentState): Promise<Partial<AgentState>> {
  if (state.needsEscalation) {
    return { answer: "This question requires clinical review.", needsEscalation: true };
    }

  if (!state.retrievedSources.length) {
    return { answer: "I could not find an applicable policy document.", needsEscalation: true };
  }

  const context = state.retrievedSources
    .map((s) => `Source ${s.id}: ${s.title}\n${s.excerpt}\nURL: ${s.url}`)
    .join("\n\n");

  const prompt = `
You are a healthcare policy assistant.
Answer using only the provided policy sources.
If the sources do not support a direct answer, say you cannot determine it.

Question:
${state.question}

Policy Sources:
${context}

Return a concise answer with citations like [policy-123].
`;

  const res = await model.invoke(prompt);
  return { answer: String(res.content) };
}

4) Wire the graph with conditional routing

This is where LangGraph earns its keep. Use StateGraph to route escalation cases away from normal generation.

const graph = new StateGraph<AgentState>()
  .addNode("classify", classifyNeed)
  .addNode("retrieve", retrievePolicies)
  
.addNode("generate", generateAnswer)
.addEdge(START, "classify")
.addConditionalEdges("classify", (state) =>
    state.needsEscalation ? END : "retrieve"
)
.addEdge("retrieve", "generate")
.addEdge("generate", END);

const app = graph.compile();

async function run() {
  const result = await app.invoke({
    question: "Is outpatient MRI covered without prior auth?",
    tenantId: "payer-a",
    region: "CA",
    userRole: "agent",
    retrievedSources: [],
   });

console.log(result.answer);
}

run();

Production Considerations

• •Auditability

Use immutable logs for every request. Store the user role, tenant ID, region, retrieved source IDs, prompt version, model version, and final answer. Healthcare operations teams will ask which policy version produced a given response.

• •Data residency

Keep retrieval indexes and inference endpoints inside approved regions. If your policies or metadata contain PHI-adjacent data, do not send them to a cross-border hosted service without explicit approval.

• •Guardrails

Add a hard rule that blocks diagnosis or treatment advice. The agent should answer coverage and policy questions only; anything clinical should route to human review or a clinical decision support workflow.

• •Monitoring

Track escalation rate, unsupported-answer rate, citation coverage, and retrieval miss rate. In healthcare, a low latency system that hallucinates is worse than a slower one that escalates correctly.

Common Pitfalls

1. •

   Mixing policy content with clinical guidance

   • •Don’t let the model infer medical recommendations from coverage language.
   • •Fix it by restricting prompts to administrative policy scope and adding an escalation branch for clinical questions.
2. •

   Ignoring metadata filters in retrieval

   • •A national plan policy is not valid for every state or line of business.
   • •Fix it by filtering on tenant, region, effective date, product type, and document status before passing sources into generation.
3. •

   Skipping citation enforcement

   • •If answers don’t point back to source text, auditors will reject them.
   • •Fix it by requiring every response to include source IDs or URLs and rejecting outputs that lack citations during post-processing.
4. •

   Storing PHI in logs

   • •Developers often log full prompts during debugging.
   • •Fix it by redacting member identifiers before logging and keeping audit trails separate from raw application logs.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.