How to Integrate Azure OpenAI for fintech with CosmosDB for production AI

Why this integration matters

If you’re building fintech AI agents, the model is only half the system. Azure OpenAI handles reasoning, extraction, summarization, and classification, while CosmosDB gives you durable state, transaction context, and low-latency retrieval for customer- or account-level memory.

That combo is what turns a chat demo into a production agent: one service generates the response, the other stores prompts, decisions, audit trails, and structured financial data you can query later.

Prerequisites

• •An Azure subscription with:
  • •Azure OpenAI resource
  • •Azure Cosmos DB for NoSQL account
• •A deployed Azure OpenAI model, such as:
  • •gpt-4o-mini
  • •gpt-4.1-mini
• •Python 3.10+
• •Installed packages:
  • •openai
  • •azure-cosmos
  • •python-dotenv
• •Environment variables set:
  • •AZURE_OPENAI_ENDPOINT
  • •AZURE_OPENAI_API_KEY
  • •AZURE_OPENAI_DEPLOYMENT
  • •AZURE_COSMOS_ENDPOINT
  • •AZURE_COSMOS_KEY
  • •AZURE_COSMOS_DATABASE
  • •AZURE_COSMOS_CONTAINER

Install dependencies:

pip install openai azure-cosmos python-dotenv

Integration Steps

1. •Set up configuration and clients.

Use environment variables. Don’t hardcode keys in your agent code.

import os
from dotenv import load_dotenv
from openai import AzureOpenAI
from azure.cosmos import CosmosClient

load_dotenv()

azure_openai_client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-06-01",
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
)

cosmos_client = CosmosClient(
    url=os.environ["AZURE_COSMOS_ENDPOINT"],
    credential=os.environ["AZURE_COSMOS_KEY"],
)

database_name = os.environ["AZURE_COSMOS_DATABASE"]
container_name = os.environ["AZURE_COSMOS_CONTAINER"]

database = cosmos_client.get_database_client(database_name)
container = database.get_container_client(container_name)
deployment_name = os.environ["AZURE_OPENAI_DEPLOYMENT"]

2. •Create a CosmosDB container for agent memory.

For fintech workloads, partition by customer or account ID. That keeps reads fast and isolates state cleanly.

from azure.cosmos import PartitionKey

# Run once during provisioning scripts
try:
    database.create_container_if_not_exists(
        id=container_name,
        partition_key=PartitionKey(path="/customerId"),
        offer_throughput=400
    )
except Exception as e:
    print(f"Container setup skipped or already exists: {e}")

A good document shape for agent memory looks like this:

memory_item = {
    "id": "session-001",
    "customerId": "cust-123",
    "conversationType": "fraud_review",
    "lastIntent": "dispute_card_charge",
    "riskFlags": ["merchant_mismatch", "high_value"],
    "updatedAt": "2026-04-21T10:00:00Z"
}

3. •Call Azure OpenAI to extract structured fintech intent.

In production, don’t just ask for prose. Ask the model to produce structured JSON you can persist in CosmosDB.

import json

prompt = """
You are a fintech support assistant.
Extract the customer's intent from the message and return JSON with keys:
intent, confidence, risk_flags, next_action.

Customer message:
"I noticed two charges from the same merchant that I don't recognize."
"""

response = azure_openai_client.chat.completions.create(
    model=deployment_name,
    messages=[
        {"role": "system", "content": "Return only valid JSON."},
        {"role": "user", "content": prompt},
    ],
    temperature=0,
)

content = response.choices[0].message.content
parsed = json.loads(content)
print(parsed)

Expected output shape:

{
  "intent": "card_charge_dispute",
  "confidence": 0.94,
  "risk_flags": ["possible_fraud", "duplicate_charge"],
  "next_action": "request_transaction_details"
}

4. •Persist model output and conversation state into CosmosDB.

Store both the raw model response and your normalized business fields. That gives you auditability and replay capability.

from datetime import datetime, timezone

document = {
    "id": "session-001",
    "customerId": "cust-123",
    "message": prompt,
    "modelResponse": parsed,
    "status": "open",
    "createdAt": datetime.now(timezone.utc).isoformat()
}

container.upsert_item(document)
print("Saved session to CosmosDB")

5. •Retrieve history and feed it back into Azure OpenAI.

This is how you build a real agent loop: read state from CosmosDB, inject it into the prompt, then write back the updated decision.

query = """
SELECT * FROM c
WHERE c.customerId = @customerId
ORDER BY c.createdAt DESC
"""

items = list(container.query_items(
    query=query,
    parameters=[{"name": "@customerId", "value": "cust-123"}],
    enable_cross_partition_query=True
))

history_summary = "\n".join([
    f"- {item['status']}: {item['modelResponse']['intent']} ({item['modelResponse']['confidence']})"
    for item in items[:5]
])

followup_response = azure_openai_client.chat.completions.create(
    model=deployment_name,
    messages=[
        {"role": "system", "content": "You are a banking operations assistant."},
        {"role": "user", "content": f"Recent customer history:\n{history_summary}\n\nWhat should we do next?"}
    ],
)

print(followup_response.choices[0].message.content)

Testing the Integration

Run an end-to-end smoke test that writes to CosmosDB, reads it back, and sends context to Azure OpenAI.

test_doc = {
    "id": "test-session-001",
    "_partitionKeyValueHint_ignored": True,
}

result = container.upsert_item({
    "id": test_doc["id"],
    "customerId": "cust-test",
    "status": "testing",
})

saved_item = container.read_item(item="test-session-001", partition_key="cust-test")

test_completion = azure_openai_client.chat.completions.create(
    model=deployment_name,
    messages=[
        {"role": "system", "content": "Return a one-line confirmation."},
        {"role": 'user', 'content': f"Cosmos item status is {saved_item['status']}."}
    ],
)

print(saved_item["status"])
print(test_completion.choices[0].message.content)

Expected output:

testing
Confirmed: CosmosDB read/write and Azure OpenAI inference are working.

Real-World Use Cases

• •

  Fraud triage agent

  • •Classify disputes with Azure OpenAI.
  • •Store case state, evidence references, and investigator notes in CosmosDB.
  • •Pull prior cases by customer or merchant pattern.
• •

  KYC/AML review assistant

  • •Extract entities from onboarding docs.
  • •Persist risk flags and review decisions.
  • •Keep an audit trail for compliance teams.
• •

  Customer servicing copilot

  • •Summarize account history before a call.
  • •Store conversation memory per customer.
  • •Generate next-best actions based on prior interactions.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.