How to Integrate Azure OpenAI for fintech with CosmosDB for AI agents

Why this integration matters

If you’re building AI agents for fintech, the model alone is not enough. You need a place to store customer context, compliance state, conversation memory, and retrieval data that survives across sessions and services.

Azure OpenAI gives you the reasoning layer. Cosmos DB gives you durable, low-latency state for agent memory, transaction context, and audit-friendly persistence.

Prerequisites

• •An Azure subscription with:
  • •Azure OpenAI resource
  • •Azure Cosmos DB account
• •A deployed Azure OpenAI model deployment name
• •Cosmos DB SQL API database and container created
• •Python 3.10+
• •Installed packages:
  • •openai
  • •azure-cosmos
  • •python-dotenv
• •Environment variables set:
  • •AZURE_OPENAI_ENDPOINT
  • •AZURE_OPENAI_API_KEY
  • •AZURE_OPENAI_DEPLOYMENT
  • •COSMOS_ENDPOINT
  • •COSMOS_KEY
  • •COSMOS_DATABASE
  • •COSMOS_CONTAINER

Integration Steps

1. •Install dependencies and load configuration

Start with a clean Python environment and wire config through environment variables. In fintech systems, hardcoding secrets is not an option.

pip install openai azure-cosmos python-dotenv

import os
from dotenv import load_dotenv

load_dotenv()

AZURE_OPENAI_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT")
AZURE_OPENAI_API_KEY = os.getenv("AZURE_OPENAI_API_KEY")
AZURE_OPENAI_DEPLOYMENT = os.getenv("AZURE_OPENAI_DEPLOYMENT")

COSMOS_ENDPOINT = os.getenv("COSMOS_ENDPOINT")
COSMOS_KEY = os.getenv("COSMOS_KEY")
COSMOS_DATABASE = os.getenv("COSMOS_DATABASE")
COSMOS_CONTAINER = os.getenv("COSMOS_CONTAINER")

2. •Initialize Azure OpenAI and Cosmos DB clients

Use the Azure OpenAI client for chat completions and the Cosmos client for persistence. This is the core wiring between reasoning and memory.

from openai import AzureOpenAI
from azure.cosmos import CosmosClient

aoai_client = AzureOpenAI(
    api_key=AZURE_OPENAI_API_KEY,
    api_version="2024-02-15-preview",
    azure_endpoint=AZURE_OPENAI_ENDPOINT,
)

cosmos_client = CosmosClient(COSMOS_ENDPOINT, credential=COSMOS_KEY)
database = cosmos_client.get_database_client(COSMOS_DATABASE)
container = database.get_container_client(COSMOS_CONTAINER)

3. •Write agent state to Cosmos DB

For an AI agent, every interaction should be persisted with metadata: user ID, intent, risk flags, timestamps, and model output. Use a deterministic partition key like userId or accountId.

from datetime import datetime, timezone
import uuid

def save_agent_turn(user_id: str, user_message: str, assistant_message: str, intent: str):
    item = {
        "id": str(uuid.uuid4()),
        "userId": user_id,
        "type": "agent_turn",
        "intent": intent,
        "userMessage": user_message,
        "assistantMessage": assistant_message,
        "createdAt": datetime.now(timezone.utc).isoformat(),
    }
    container.upsert_item(item)
    return item

That upsert_item call is the standard Cosmos DB write path here. If you’re storing conversation memory or case notes, keep the document shape stable so downstream retrieval stays predictable.

4. •Call Azure OpenAI with fintech-specific context

Send structured context from Cosmos into the model prompt. In fintech workflows, that usually means recent interactions, account status summaries, or policy constraints.

def generate_response(user_id: str, message: str):
    history_query = """
    SELECT TOP 5 c.userMessage, c.assistantMessage
    FROM c
    WHERE c.userId = @userId AND c.type = 'agent_turn'
    ORDER BY c.createdAt DESC
    """

    history = list(container.query_items(
        query=history_query,
        parameters=[{"name": "@userId", "value": user_id}],
        enable_cross_partition_query=True
    ))

    context_lines = []
    for turn in reversed(history):
        context_lines.append(f"User: {turn['userMessage']}")
        context_lines.append(f"Assistant: {turn['assistantMessage']}")

    messages = [
        {
            "role": "system",
            "content": (
                "You are a fintech support agent. "
                "Do not provide investment advice. "
                "Escalate suspicious activity or compliance-sensitive requests."
            )
        },
        {
            "role": "user",
            "content": "\n".join(context_lines + [f"User: {message}"])
        }
    ]

    response = aoai_client.chat.completions.create(
        model=AZURE_OPENAI_DEPLOYMENT,
        messages=messages,
        temperature=0.2,
    )

    assistant_message = response.choices[0].message.content
    save_agent_turn(user_id, message, assistant_message, intent="customer_support")
    return assistant_message

This pattern works because Cosmos DB handles retrieval of prior state while Azure OpenAI handles natural language generation. For regulated workflows, keep temperature low and constrain the system prompt tightly.

5. •Wrap it in an agent handler

Your application layer should expose one function that receives input, calls retrieval if needed, invokes the model, then persists the result. That keeps orchestration clean.

def handle_agent_request(user_id: str, message: str):
    reply = generate_response(user_id=user_id, message=message)
    return {
        "userId": user_id,
        "reply": reply,
        "status": "ok"
    }

Testing the Integration

Run a simple end-to-end test that sends a message, stores it in Cosmos DB, then reads it back.

if __name__ == "__main__":
    result = handle_agent_request(
        user_id="cust_1001",
        message="Can you summarize my last support request?"
    )

    print(result["status"])
    print(result["reply"])

    stored_items = list(container.query_items(
        query="SELECT TOP 1 * FROM c WHERE c.userId = @userId ORDER BY c.createdAt DESC",
        parameters=[{"name": "@userId", "value": "cust_1001"}],
        enable_cross_partition_query=True
    ))

    print(stored_items[0]["type"])

Expected output:

ok
[assistant summary about the last support request]
agent_turn

If that works end to end, your agent can now reason with Azure OpenAI and persist memory in Cosmos DB.

Real-World Use Cases

• •

  Customer support agents with persistent memory

  • •Store prior complaints, KYC status notes, and resolution history in Cosmos DB.
  • •Use Azure OpenAI to draft responses based on current case context.
• •

  Fraud triage assistants

  • •Persist suspicious patterns and analyst decisions.
  • •Let the agent summarize alerts and recommend escalation paths without losing audit trail.
• •

  Personal finance copilots

  • •Keep user preferences, spending summaries, and product eligibility snapshots in Cosmos DB.
  • •Use Azure OpenAI to explain account activity or guide users through next steps without exposing raw backend complexity.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.