How to Integrate Azure OpenAI for retail banking with CosmosDB for AI agents

Azure OpenAI gives you the reasoning layer for banking conversations. CosmosDB gives you the durable state layer for customer profiles, consent, interaction history, and agent memory. Put them together and you can build retail banking agents that answer product questions, summarize account activity, and carry context across sessions without stuffing everything into prompts.

Prerequisites

• •Azure subscription with:
  • •Azure OpenAI resource deployed
  • •A model deployment name for chat completions
• •Azure Cosmos DB account with:
  • •Database created
  • •Container created for agent state or customer memory
• •Python 3.10+
• •Installed packages:
  • •openai
  • •azure-cosmos
  • •python-dotenv
• •Environment variables set:
  • •AZURE_OPENAI_ENDPOINT
  • •AZURE_OPENAI_API_KEY
  • •AZURE_OPENAI_DEPLOYMENT
  • •COSMOS_ENDPOINT
  • •COSMOS_KEY
  • •COSMOS_DATABASE
  • •COSMOS_CONTAINER

Install dependencies:

pip install openai azure-cosmos python-dotenv

Integration Steps

1) Initialize Azure OpenAI and CosmosDB clients

Start by creating both clients in the same service. Keep them in a thin integration layer so your agent logic stays separate from persistence.

import os
from dotenv import load_dotenv
from openai import AzureOpenAI
from azure.cosmos import CosmosClient

load_dotenv()

azure_openai_client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-02-15-preview",
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
)

cosmos_client = CosmosClient(
    url=os.environ["COSMOS_ENDPOINT"],
    credential=os.environ["COSMOS_KEY"]
)

database = cosmos_client.get_database_client(os.environ["COSMOS_DATABASE"])
container = database.get_container_client(os.environ["COSMOS_CONTAINER"])

deployment_name = os.environ["AZURE_OPENAI_DEPLOYMENT"]

2) Store customer context in CosmosDB

For retail banking, you usually need a compact profile: customer tier, product holdings, consent flags, and recent intent. Use the customer ID as the partition key so reads stay cheap.

from datetime import datetime, timezone

def upsert_customer_state(customer_id: str, profile: dict):
    item = {
        "id": customer_id,
        "customerId": customer_id,
        "profile": profile,
        "updatedAt": datetime.now(timezone.utc).isoformat()
    }
    container.upsert_item(item)
    return item

customer_profile = {
    "name": "Amina Patel",
    "segment": "mass_affluent",
    "products": ["checking", "credit_card"],
    "consent": {
        "personalized_offers": True,
        "transaction_insights": True
    },
    "recent_intent": "ask about overdraft fee"
}

upsert_customer_state("cust_10021", customer_profile)

3) Retrieve memory before calling Azure OpenAI

Pull stored context from CosmosDB and inject only what the model needs. Don’t dump the whole document into the prompt; keep it focused on banking-relevant facts.

def get_customer_state(customer_id: str) -> dict | None:
    try:
        return container.read_item(item=customer_id, partition_key=customer_id)
    except Exception:
        return None

state = get_customer_state("cust_10021")

system_prompt = """
You are a retail banking assistant.
Use only the provided customer context.
Do not invent balances, fees, or account activity.
If data is missing, ask a follow-up question.
"""

user_message = "Can you explain whether I would be charged an overdraft fee?"
context_block = f"Customer context: {state['profile']}" if state else "Customer context: unavailable"

4) Call Azure OpenAI with retrieved bank context

Use chat.completions.create() to generate the response. This is where the model turns structured banking data into a useful answer.

response = azure_openai_client.chat.completions.create(
    model=deployment_name,
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": f"{context_block}\n\nQuestion: {user_message}"}
    ],
    temperature=0.2,
)

answer = response.choices[0].message.content
print(answer)

5) Persist the interaction back to CosmosDB

After each turn, write the interaction summary back to CosmosDB. In production, this becomes your agent memory and audit trail.

from datetime import datetime, timezone

def save_interaction(customer_id: str, user_text: str, assistant_text: str):
    record = {
        "id": f"{customer_id}:{datetime.now(timezone.utc).timestamp()}",
        "customerId": customer_id,
        "type": "agent_interaction",
        "userText": user_text,
        "assistantText": assistant_text,
        "createdAt": datetime.now(timezone.utc).isoformat()
    }
    container.upsert_item(record)
    return record

save_interaction("cust_10021", user_message, answer)

Testing the Integration

Run a full round trip: load state from CosmosDB, send it to Azure OpenAI, then persist the result.

test_customer_id = "cust_10021"

state = get_customer_state(test_customer_id)
prompt = f"""
Customer profile: {state['profile']}
Question: What should I know about overdraft fees on my checking account?
"""

result = azure_openai_client.chat.completions.create(
    model=deployment_name,
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": prompt}
    ],
)

reply = result.choices[0].message.content
save_interaction(test_customer_id, prompt, reply)

print("MODEL_REPLY:")
print(reply)

stored = get_customer_state(test_customer_id)
print("\nCOSMOS_CHECK:")
print(stored["customerId"])

Expected output:

MODEL_REPLY:
Based on your checking account context, I can explain how overdraft fees typically work...
COSMOS_CHECK:
cust_10021

Real-World Use Cases

• •

  Personalized banking assistants

  • •Answer questions using customer-specific profile data stored in CosmosDB.
  • •Keep responses grounded in consented data only.
• •

  Agent memory for service workflows

  • •Track prior complaints, KYC status, card replacement requests, and escalation history.
  • •Let multiple agents share state without duplicating logic.
• •

  Next-best-action banking journeys

  • •Use Azure OpenAI to summarize intent and generate outreach text.
  • •Use CosmosDB to persist campaign eligibility and interaction outcomes for later retrieval.

If you want this production-ready, add retrieval filters for consent, encrypt sensitive fields at rest in your app layer where required by policy, and separate transactional account data from agent memory. For retail banking systems, that boundary matters more than prompt quality.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.