How to Integrate Azure OpenAI for retail banking with CosmosDB for production AI

Combining Azure OpenAI with Cosmos DB gives you a practical pattern for retail banking agents: structured customer data stays in a durable operational store, while the model handles intent extraction, summarization, and response drafting. That means you can build assistants for account servicing, dispute intake, loan pre-qualification, and branch support without stuffing sensitive state into prompts.

The useful part is not “chat with a database.” It’s controlled orchestration: retrieve the right customer context from Cosmos DB, pass only the minimum necessary fields into Azure OpenAI, then persist the agent’s decision trail back to Cosmos DB for audit and follow-up.

Prerequisites

• •An Azure subscription with:
  • •Azure OpenAI resource deployed
  • •Azure Cosmos DB for NoSQL account created
• •Python 3.10+
• •Installed packages:
  • •azure-ai-openai
  • •azure-cosmos
  • •python-dotenv
• •An Azure OpenAI deployment name for a chat model, such as gpt-4o-mini
• •Cosmos DB database and container ready for:
  • •customer profiles
  • •conversation state
  • •case notes
• •Environment variables set:
  • •AZURE_OPENAI_ENDPOINT
  • •AZURE_OPENAI_API_KEY
  • •AZURE_OPENAI_DEPLOYMENT
  • •COSMOS_ENDPOINT
  • •COSMOS_KEY
  • •COSMOS_DATABASE_NAME
  • •COSMOS_CONTAINER_NAME

Integration Steps

1. •Install dependencies and load configuration.

pip install azure-ai-openai azure-cosmos python-dotenv

import os
from dotenv import load_dotenv

load_dotenv()

AZURE_OPENAI_ENDPOINT = os.environ["AZURE_OPENAI_ENDPOINT"]
AZURE_OPENAI_API_KEY = os.environ["AZURE_OPENAI_API_KEY"]
AZURE_OPENAI_DEPLOYMENT = os.environ["AZURE_OPENAI_DEPLOYMENT"]

COSMOS_ENDPOINT = os.environ["COSMOS_ENDPOINT"]
COSMOS_KEY = os.environ["COSMOS_KEY"]
COSMOS_DATABASE_NAME = os.environ["COSMOS_DATABASE_NAME"]
COSMOS_CONTAINER_NAME = os.environ["COSMOS_CONTAINER_NAME"]

2. •Create Cosmos DB client access for customer state and case storage.

from azure.cosmos import CosmosClient, PartitionKey

cosmos_client = CosmosClient(COSMOS_ENDPOINT, credential=COSMOS_KEY)
database = cosmos_client.get_database_client(COSMOS_DATABASE_NAME)
container = database.get_container_client(COSMOS_CONTAINER_NAME)

# Example document shape:
# {
#   "id": "cust_1001",
#   "pk": "cust_1001",
#   "customer_name": "Amina Patel",
#   "account_status": "active",
#   "recent_interactions": [],
#   "risk_flags": ["none"]
# }
customer = container.read_item(item="cust_1001", partition_key="cust_1001")
print(customer["customer_name"])

3. •Call Azure OpenAI with retrieved banking context.

For retail banking, keep prompts narrow. Pull only what the agent needs: customer name, account status, recent interaction summary, and the user request.

from azure.ai.openai import OpenAIClient
from azure.core.credentials import AzureKeyCredential

openai_client = OpenAIClient(
    endpoint=AZURE_OPENAI_ENDPOINT,
    credential=AzureKeyCredential(AZURE_OPENAI_API_KEY)
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a retail banking service assistant. "
            "Do not invent policy. "
            "If the request needs human review, say so clearly."
        ),
    },
    {
        "role": "user",
        "content": (
            f"Customer: {customer['customer_name']}\n"
            f"Account status: {customer['account_status']}\n"
            f"Recent flags: {', '.join(customer.get('risk_flags', []))}\n"
            f"Request: The customer wants to dispute a card transaction."
        ),
    },
]

response = openai_client.get_chat_completions(
    deployment_id=AZURE_OPENAI_DEPLOYMENT,
    messages=messages,
    temperature=0.2,
    max_tokens=300,
)

assistant_text = response.choices[0].message.content
print(assistant_text)

4. •Persist the AI result back into Cosmos DB for traceability.

This is where production systems differ from demos. Store the model output, timestamps, and a small structured summary so downstream workflows can pick it up.

from datetime import datetime, timezone

case_note = {
    "id": f"case_{customer['id']}_{datetime.now(timezone.utc).timestamp()}",
    "pk": customer["id"],
    "customer_id": customer["id"],
    "created_at": datetime.now(timezone.utc).isoformat(),
    "source": "azure-openai",
    "request_type": "card_dispute",
    "summary": assistant_text,
    "status": "needs_review"
}

container.create_item(body=case_note)

customer.setdefault("recent_interactions", []).append({
    "timestamp": datetime.now(timezone.utc).isoformat(),
    "channel": "agent",
    "topic": "card_dispute",
})

container.upsert_item(customer)

5. •Add a simple orchestration function to tie retrieval, generation, and persistence together.

def handle_banking_request(customer_id: str, user_request: str) -> str:
    profile = container.read_item(item=customer_id, partition_key=customer_id)

    messages = [
        {
            "role": "system",
            "content": (
                "You are a retail banking assistant. "
                "Use concise language. "
                "Escalate disputes, fraud claims, and complaints when policy requires."
            ),
        },
        {
            "role": "user",
            "content": (
                f"Customer profile:\n"
                f"- Name: {profile['customer_name']}\n"
                f"- Status: {profile['account_status']}\n"
                f"- Flags: {profile.get('risk_flags', [])}\n\n"
                f"User request:\n{user_request}"
            ),
        },
    ]

    result = openai_client.get_chat_completions(
        deployment_id=AZURE_OPENAI_DEPLOYMENT,
        messages=messages,
        temperature=0.2,
        max_tokens=250,
    )

    answer = result.choices[0].message.content

    container.create_item({
        "id": f"audit_{customer_id}_{datetime.now(timezone.utc).timestamp()}",
        "pk": customer_id,
        "type": "audit_log",
        "request": user_request,
        "response": answer,
        "created_at": datetime.now(timezone.utc).isoformat(),
    })

    return answer


print(handle_banking_request("cust_1001", 
      "I need help disputing a debit card charge from yesterday."))

Testing the Integration

Run an end-to-end check against one known customer record in Cosmos DB.

test_customer_id = "cust_1001"
test_request = (
    "Customer says they do not recognize a debit card charge of $48.22 "
    f"and wants next steps."
)

result_text = handle_banking_request(test_customer_id, test_request)
print("MODEL OUTPUT:")
print(result_text)

stored_case = list(
    container.query_items(
        query="SELECT TOP 1 * FROM c WHERE c.pk = @pk AND c.type != 'audit_log' ORDER BY c.created_at DESC",
        parameters=[{"name": "@pk", "value": test_customer_id}],
        enable_cross_partition_query=True,
    )
)
print("CASE STORED:", len(stored_case) > 0)

Expected output:

MODEL OUTPUT:
I can help route this as a card dispute...
CASE STORED: True

If that prints cleanly and the item appears in Cosmos DB, your integration is working end to end.

Real-World Use Cases

• •

  Card dispute triage

  • •Retrieve customer history from Cosmos DB.
  • •Use Azure OpenAI to classify whether the issue is fraud, merchant dispute, or service complaint.
  • •Store the generated case summary for operations review.
• •

  Loan servicing assistant

  • •Pull application status and missing documents from Cosmos DB.
  • •Have Azure OpenAI draft next-step instructions for the customer.
  • •Persist follow-up tasks back into Cosmos DB for workflow automation.
• •

  Branch support copilot

  • •Fetch recent interactions and product holdings.
  • •Generate guided responses for branch staff handling account changes.
  • •Keep an audit trail of every recommendation in Cosmos DB.

The production pattern is simple: Cosmos DB owns state, Azure OpenAI owns language reasoning, and your application owns policy enforcement. Keep those boundaries tight and your retail banking agent will be much easier to operate under real compliance constraints.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.