What is model routing in AI Agents? A Guide for compliance officers in retail banking

Model routing is the process of sending a user request to the most appropriate AI model based on the task, risk level, policy rules, or cost. In AI agents, model routing decides whether a prompt should go to a small fast model, a larger reasoning model, or a specialized model for things like retrieval, classification, or summarization.

How It Works

Think of model routing like a bank’s internal approval chain.

A teller does not handle every request the same way. A simple balance inquiry gets handled immediately. A mortgage exception, fraud concern, or sanctions-related issue gets escalated to the right specialist. Model routing works the same way: the agent inspects the request first, then sends it to the model best suited for that job.

In practice, the routing layer looks at signals such as:

• •The type of request
  • •Example: FAQ answer, complaint triage, KYC summary, transaction explanation
• •The sensitivity of the content
  • •Example: personal data, account details, adverse action reasons
• •The required quality level
  • •Example: simple classification versus multi-step reasoning
• •Policy constraints
  • •Example: high-risk financial advice must use stricter controls
• •Cost and latency targets
  • •Example: use a cheaper model for routine tasks and reserve heavier models for complex cases

A common setup is:

1. •A user asks the agent a question.
2. •A router classifies the request.
3. •The router applies business and compliance rules.
4. •The request is sent to the selected model.
5. •The agent logs which model was used and why.

For compliance teams, that last step matters. If a customer complaint is handled by one model and an AML-related query is handled by another, you need traceability. You want to know which model made which decision path, what data it saw, and whether policy rules were followed.

A simple analogy: imagine a mailroom in head office.

• •Standard letters go to general processing.
• •Legal notices go to legal.
• •Fraud reports go to financial crime.
• •Executive correspondence goes to senior management.

The mailroom is not “making decisions” about content quality; it is routing based on rules. That is what good AI routing does in regulated environments.

Why It Matters

Compliance officers in retail banking should care because routing changes how AI risk is managed at scale.

• •

  It reduces unnecessary exposure of sensitive data
  Not every request needs to hit the most powerful model. Routing can keep low-risk tasks on smaller models with tighter controls.

• •

  It supports policy enforcement
  A router can block certain requests from being handled by models that are not approved for regulated outputs or customer-facing advice.

• •

  It improves auditability
  If an AI agent routes different cases differently, you need logs showing why each path was chosen. That helps with internal review and regulatory exams.

• •

  It helps with consistency in customer treatment
  Similar cases should follow similar paths. Routing rules can reduce random variation in how customer queries are handled.

• •

  It lowers operational risk
  High-risk requests can be escalated to human review or stricter models instead of being answered automatically by a general-purpose system.

Here is the key point: model routing is not just an engineering optimization. In banking, it is part of control design.

If you do not govern routing properly, you can end up with:

• •Sensitive data going to an unapproved vendor/model
• •Customer-facing responses generated by an underpowered model
• •Different treatment of similar cases without clear justification
• •Weak audit trails when something goes wrong

Real Example

A retail bank deploys an AI agent for customer service across mobile app chat and call center support.

The agent handles three common requests:

• •“What is my current checking balance?”
• •“Why was my card declined at checkout?”
• •“I think this charge may be fraudulent.”

The bank sets up routing like this:

Here is how it plays out:

1. •The customer asks about a declined card payment.
2. •The router checks intent and sees it may involve authorization logic.
3. •It sends the case to a stronger reasoning model that can inspect recent transaction metadata.
4. •The agent drafts an explanation like: “The transaction was declined because it exceeded your daily limit.”
5. •Before sending it back, the system checks whether the response includes any restricted details or unsupported claims.
6. •If fraud indicators appear instead of a normal decline reason, the route changes again and creates a case for human review.

From a compliance perspective, this setup has clear benefits:

• •Routine factual answers stay constrained
• •Sensitive issues get higher scrutiny
• •Fraud-related cases do not get fully automated without controls
• •Every route can be logged and reviewed

This is also where banks often add guardrails:

• •Approved response templates for regulated language
• •PII redaction before sending text to external models
• •Human-in-the-loop review for escalations
• •Monitoring for route drift over time

The important lesson: routing is part of your control framework, not just your AI architecture diagram.

Related Concepts

• •

  Prompt classification
  The first step in deciding what kind of request came in.

• •

  Human-in-the-loop escalation
  Sending certain cases to staff instead of letting the agent answer directly.

• •

  Model governance
  Policies around which models are approved for which use cases.

• •

  PII redaction
  Removing personal data before sending text to a model provider.

• •

  Audit logging
  Recording which route was taken, what data was used, and what output was produced.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.