What is prompt engineering in AI Agents? A Guide for product managers in retail banking

Prompt engineering is the practice of designing the instructions, context, and constraints you give an AI model so it produces the output you want. In AI agents, prompt engineering is how you shape what the agent should do, what data it should use, what it should avoid, and how it should respond.

How It Works

Think of an AI agent like a junior branch operations assistant.

If you say, “Help this customer,” you’ll get something vague. If you say, “Check whether this debit card dispute qualifies for provisional credit, ask for the transaction date if missing, then summarize the next step in plain language,” you’re giving the assistant a job description, decision rules, and a response format.

That is prompt engineering.

In retail banking, an AI agent usually sits between a customer or employee and several systems:

• •CRM
• •core banking
• •card disputes
• •KYC/AML checks
• •knowledge base articles
• •ticketing tools

The prompt tells the agent:

• •Role: what kind of assistant it is
• •Task: what outcome it should produce
• •Context: relevant policy, customer data, or case history
• •Constraints: what it must not do
• •Format: how to structure the response

A simple mental model:

A useful analogy is a teller tray at a branch.

If the tray has no labels, papers get mixed up. If each slot is labeled — cash deposit slips here, loan docs there, exception cases here — work moves faster and errors drop. Prompt engineering does the same thing for an AI agent: it organizes the work before the model starts reasoning.

For product managers, this matters because prompts are not just “wording.” They are product behavior definitions. A small change in instruction can change:

• •whether the agent asks follow-up questions
• •whether it escalates to a human
• •whether it gives a policy-safe answer
• •whether downstream systems can consume its output

Why It Matters

Prompt engineering matters in retail banking because:

• •

  It controls risk

  Bad prompts can cause hallucinated answers about fees, overdrafts, disputes, or eligibility. Good prompts keep the agent inside policy boundaries.

• •

  It improves resolution rates

  Clear instructions help agents collect missing details upfront and route cases correctly on the first pass.

• •

  It reduces operational load

  Well-designed prompts can turn messy customer messages into structured case notes, saving time for service teams and back-office staff.

• •

  It makes outputs usable by systems

  If your agent needs to create tickets or trigger workflows, the prompt must force consistent formats like JSON or templated summaries.

For PMs in banking, this is where product design meets control design. You are not just asking “Can the model answer?” You are asking:

• •Can it answer safely?
• •Can ops trust it?
• •Can compliance review it?
• •Can we measure quality?

Real Example

Let’s say your bank wants an AI agent to help with credit card dispute intake in mobile banking.

A weak prompt would be:

Help customers with card disputes.

That sounds fine until the model starts giving inconsistent advice, missing key details, or using language that sounds like final adjudication.

A better prompt looks more like this:

You are a retail banking dispute intake assistant.

Your job:
1. Identify whether the customer is reporting fraud, merchant dispute, duplicate charge, or unknown.
2. Ask for missing required details only if needed:
   - transaction date
   - merchant name
   - amount
   - whether the card was present
3. Do not promise chargeback approval.
4. Do not mention internal policy text.
5. If fraud is suspected and card may be compromised, recommend card replacement and escalation.
6. Output only valid JSON with these fields:
   - dispute_type
   - missing_fields
   - recommended_next_step
   - customer_message

Now imagine a customer says:

I see two charges from Hotel Central for $240 on my debit card last Friday.

A good agent response might be:

{
  "dispute_type": "duplicate_charge",
  "missing_fields": [],
  "recommended_next_step": "Create dispute case and confirm transaction review",
  "customer_message": "I can help with that duplicate charge. I’ve captured the merchant name and amount. Your next step is to review both transactions and submit the dispute for investigation."
}

Why this works:

• •It classifies the issue instead of guessing.
• •It avoids overpromising outcomes.
• •It produces structured output that a workflow engine can use.
• •It keeps customer language simple and compliant.

For a PM, this means prompt engineering is part of your operating model. You define what good looks like before engineering builds orchestration around it.

Related Concepts

Here are adjacent topics worth knowing:

• •

  System prompts

  The top-level instructions that define agent behavior across many interactions.

• •

  Few-shot prompting

  Giving examples inside the prompt so the model learns patterns from sample inputs and outputs.

• •

  Tool calling

  Letting an AI agent call APIs or internal services instead of guessing from text alone.

• •

  Guardrails

  Rules that block unsafe actions or responses, especially important in regulated banking workflows.

• •

  RAG (Retrieval-Augmented Generation)

  A pattern where the agent pulls facts from approved sources before answering instead of relying on memory alone.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.