What is model routing in AI Agents? A Guide for compliance officers in banking

Model routing is the process of sending each AI request to the most appropriate model based on rules, risk, cost, latency, or task type. In AI agents, model routing decides whether a request goes to a small fast model, a larger more capable model, or a specialized model for a specific control or domain.

How It Works

Think of model routing like a bank’s internal case triage desk.

A simple customer query, like “What is my mortgage balance?”, should not go through the same path as “Summarize this suspicious transaction pattern and explain why it may trigger SAR review.” The first can be handled by a cheaper, faster model. The second may need a stronger reasoning model, plus tighter logging and human review.

In practice, an AI agent usually has three parts:

• •Input classifier or router
  • •Reads the user request
  • •Detects intent, sensitivity, complexity, and policy constraints
• •Routing rules
  • •Decide which model to use
  • •May consider data classification, jurisdiction, language, risk tier, or cost
• •Target model
  • •Executes the task
  • •Could be a general-purpose LLM, a domain-tuned model, or a workflow-specific tool

A compliance-friendly way to think about it is this:

The key point is that routing is not just about performance. It is also about control.

For example:

• •Low-risk customer FAQ → small model
• •Regulated advice or product suitability question → stronger guardrails and possibly human approval
• •PII-heavy document analysis → approved secure model only
• •Fraud or AML triage → specialized workflow with strict logging

A well-designed router can also enforce policy before any prompt reaches a model. That matters because compliance does not want every request treated equally. A KYC document summary and a marketing email draft should not follow the same data path.

Why It Matters

Compliance officers should care because model routing changes how risk is managed in production.

• •It controls where sensitive data goes
  • •Routing can prevent PII, account data, or customer complaints from reaching models that are not approved for that data class.
• •It supports policy-based escalation
  • •High-risk requests can be sent to more controlled models or into human review instead of being answered automatically.
• •It helps with auditability
  • •If routing decisions are logged, you can show why a specific request used a particular model.
• •It reduces unnecessary exposure
  • •Not every request needs the most powerful model. Routing can limit overuse of broad models on regulated tasks.
• •It makes governance practical
  • •Different models can have different approvals, retention rules, and vendor terms. Routing lets you apply those rules consistently.

The compliance concern is not “which model is best?” It is “which model is allowed for this use case under our controls?”

That distinction matters in banking because one agent may handle both low-risk service requests and high-risk financial communications. Without routing, the system becomes hard to govern.

Real Example

Consider a retail bank deploying an AI agent inside its customer service portal.

The agent handles three common requests:

• •“Reset my online banking password”
• •“Explain why my card payment was declined”
• •“Review this complaint about an unauthorized transfer”

A basic routing setup might work like this:

1. •

   Password reset

   • •Routed to a small low-latency support model
   • •No sensitive financial reasoning needed
   • •Response uses predefined workflow steps
2. •

   Card payment declined

   • •Routed to a mid-tier customer support model
   • •Can explain standard decline reasons from approved knowledge base content
   • •Still constrained from making account-specific claims unless authenticated
3. •

   Unauthorized transfer complaint

   • •Routed to a higher-control workflow
   • •The agent extracts facts but does not decide liability
   • •Case is escalated to fraud operations or complaints handling
   • •Full audit trail is stored for review

Why route it this way?

Because the third case touches conduct risk, dispute handling, possible fraud indicators, and regulatory deadlines. A generic response generator should not improvise there.

In this setup, the router might use rules like:

IF request contains keywords: "unauthorized", "fraud", "complaint", "chargeback"
THEN route to high-control workflow + human review

IF request contains account balance OR transaction history AND user is authenticated
THEN route to secure customer-service model

IF request asks for general product info with no personal data
THEN route to low-cost FAQ model

This gives compliance teams something they can actually govern:

• •Approved paths for each category of request
• •Clear escalation triggers
• •Model-specific logging and retention rules
• •Reduced chance of an uncontrolled answer being sent to customers

Related Concepts

• •Model governance
  • •The policies and controls around which models are approved for which tasks.
• •Prompt filtering
  • •Pre-processing user input to block unsafe content or sensitive data before it reaches a model.
• •Human-in-the-loop
  • •Requiring staff review for certain outputs before they are sent externally.
• •Risk tiering
  • •Classifying requests by sensitivity so controls match the level of exposure.
• •Audit logging
  • •Recording prompts, routing decisions, outputs, and approvals for review and investigation.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.