How to Build a customer support Agent Using CrewAI in Python for healthcare

A healthcare customer support agent handles patient questions about appointments, billing, pre-authorizations, medication refill status, and clinic policies without exposing protected health information to the wrong place. It matters because support volume is high, response time affects patient experience, and every interaction has compliance risk if you let an LLM improvise around PHI.

Architecture

• •

  Intake layer

  • •Receives chat, email, or portal messages.
  • •Normalizes the request into a structured ticket with patient-safe metadata only.
• •

  Triage agent

  • •Classifies intent: scheduling, billing, benefits, prescription status, escalation.
  • •Detects urgent cases like symptoms or adverse events and routes them away from support.
• •

  Knowledge retrieval tool

  • •Pulls from approved sources only: policy docs, FAQ pages, SOPs, payer rules.
  • •Never queries raw EHR data unless you have explicit authorization and a compliant integration.
• •

  Response drafting agent

  • •Generates a short answer grounded in the retrieved policy.
  • •Uses strict guardrails to avoid medical advice and PHI leakage.
• •

  Escalation workflow

  • •Creates a handoff for human staff when confidence is low or the request touches clinical content.
  • •Attaches audit metadata: intent, source docs used, timestamp, model version.
• •

  Logging and compliance layer

  • •Stores prompts, outputs, tool calls, and redaction events.
  • •Supports retention policies, audit review, and data residency constraints.

Implementation

1) Install CrewAI and define your support tools

Use tools to keep the agent grounded in approved healthcare content. For production, these tools should query controlled sources like a policy database or a document index with access controls.

from crewai import Agent, Task, Crew
from crewai.tools import BaseTool
from pydantic import BaseModel
from typing import Type

class PolicyLookupInput(BaseModel):
    query: str

class HealthcarePolicyLookupTool(BaseTool):
    name: str = "healthcare_policy_lookup"
    description: str = "Search approved healthcare support policies and FAQs."
    args_schema: Type[BaseModel] = PolicyLookupInput

    def _run(self, query: str) -> str:
        # Replace with vector DB / search index backed by approved documents.
        policies = {
            "appointment": "Patients can reschedule appointments up to 24 hours in advance.",
            "billing": "Billing questions are handled by the revenue cycle team during business hours.",
            "refill": "Prescription refill requests are routed to the pharmacy team."
        }
        for key, value in policies.items():
            if key in query.lower():
                return value
        return "No matching policy found. Escalate to a human support specialist."

policy_tool = HealthcarePolicyLookupTool()

2) Create specialized agents with narrow responsibilities

Do not build one giant agent that does everything. Split triage from response drafting so each step is auditable and easier to constrain.

triage_agent = Agent(
    role="Healthcare Support Triage Specialist",
    goal="Classify incoming patient support requests and detect escalation triggers.",
    backstory=(
        "You work in a regulated healthcare contact center. "
        "You must never provide clinical advice or expose PHI unnecessarily."
    ),
    tools=[policy_tool],
    verbose=True,
)

response_agent = Agent(
    role="Healthcare Support Response Writer",
    goal="Draft concise patient-safe responses using approved policy text only.",
    backstory=(
        "You write compliant customer support responses for healthcare operations. "
        "You avoid diagnosis, treatment guidance, and unsupported claims."
    ),
    tools=[policy_tool],
    verbose=True,
)

3) Define tasks that enforce the workflow

The first task classifies the request. The second task drafts the reply using only the triage output and policy lookup result.

triage_task = Task(
    description=(
        "Review this patient message and classify it into one of these categories: "
        "appointment, billing, refill, benefits, clinical_escalation, other. "
        "If the message mentions symptoms, pain, medication side effects, or emergencies, "
        "mark it as clinical_escalation."
    ),
    expected_output="A JSON-like summary with category, urgency flag, and escalation reason if any.",
    agent=triage_agent,
)

response_task = Task(
    description=(
        "Using the triage result and approved policy text only, draft a short support reply. "
        "Do not mention internal systems. Do not include medical advice. "
        "If escalation is required, tell the patient a human specialist will follow up."
    ),
    expected_output="A patient-safe response under 120 words.",
    agent=response_agent,
)

4) Run the crew with explicit inputs

CrewAI’s Crew object coordinates execution. In production you would pass redacted input text here after your PHI filter runs upstream.

crew = Crew(
    agents=[triage_agent, response_agent],
    tasks=[triage_task, response_task],
    verbose=True,
)

result = crew.kickoff(inputs={
    "patient_message": (
        "Hi, I need to move my cardiology appointment next week "
        "and also check whether my insurance covers it."
    )
})

print(result)

A better production pattern is to preprocess incoming text before it reaches the crew:

• •Redact names, member IDs, phone numbers where possible.
• •Detect emergency language and bypass automation.
• •Attach tenant ID and region so routing respects residency rules.
• •Store an immutable trace of tool usage for audit review.

Production Considerations

• •

  Compliance

  • •Treat every prompt as potentially sensitive.
  • •Run PHI detection before model invocation and block unsupported requests.
  • •Keep your prompts aligned with HIPAA minimum necessary principles.
• •

  Auditability

  • •Log task inputs/outputs, tool calls, model version, and timestamps.
  • •Make logs immutable or append-only for investigation and compliance review.
  • •Preserve evidence of which policy document was used to answer each question.
• •

  Data residency

  • •Route workloads by tenant or region if your organization operates across jurisdictions.
  • •Keep embeddings/vector stores inside approved cloud regions.
  • •Avoid sending support transcripts to third-party services without a DPA and security review.
• •

  Guardrails

  • •Add hard rules for symptoms, self-harm language, medication side effects, and emergency terms.
  • •Escalate anything clinical to licensed staff immediately.
  • •Restrict tools so agents can only read approved knowledge sources.

Common Pitfalls

1. •

   Letting the agent answer clinical questions

   • •Mistake: The agent starts giving symptom advice because it “sounds helpful.”
   • •Fix: Hard-code escalation rules for anything diagnostic or treatment-related.
2. •

   Passing raw PHI into prompts

   • •Mistake: Full chat transcripts get sent directly to the LLM stack.
   • •Fix: Redact identifiers upstream and pass only what is required for routing.
3. •

   Using ungoverned knowledge sources

   • •Mistake: The agent searches public web results or stale PDFs from shared drives.
   • •Fix: Restrict retrieval to versioned policy documents with ownership and review dates.
4. •

   Skipping audit trails

   • •Mistake: Support replies are generated without traceability.
   • •Fix: Persist task inputs/outputs plus tool call metadata so compliance can reconstruct decisions later.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.