How to Integrate Azure OpenAI for banking with CosmosDB for startups

Combining Azure OpenAI for banking with CosmosDB gives you a clean pattern for building regulated AI agents that can answer questions, retrieve customer context, and persist conversation state without turning your app into a pile of glue code. For startups, this is the practical path to shipping an assistant that can handle banking workflows like account summaries, policy Q&A, fraud triage, and support case drafting while keeping data in a structured store.

Prerequisites

• •An Azure subscription with:
  • •Azure OpenAI resource deployed
  • •Cosmos DB account created
• •A deployed Azure OpenAI model deployment name, such as:
  • •gpt-4o-mini
  • •gpt-4.1-mini
• •A Cosmos DB database and container ready for agent memory or customer records
• •Python 3.10+
• •Installed packages:
  • •openai
  • •azure-cosmos
  • •python-dotenv
• •Environment variables configured:
  • •AZURE_OPENAI_ENDPOINT
  • •AZURE_OPENAI_API_KEY
  • •AZURE_OPENAI_DEPLOYMENT
  • •COSMOS_ENDPOINT
  • •COSMOS_KEY
  • •COSMOS_DATABASE_NAME
  • •COSMOS_CONTAINER_NAME

Integration Steps

1. •

   Install dependencies and load configuration

   Keep secrets out of code. Use environment variables and load them from .env during local development.

   import os
   from dotenv import load_dotenv
   
   load_dotenv()
   
   AZURE_OPENAI_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT")
   AZURE_OPENAI_API_KEY = os.getenv("AZURE_OPENAI_API_KEY")
   AZURE_OPENAI_DEPLOYMENT = os.getenv("AZURE_OPENAI_DEPLOYMENT")
   
   COSMOS_ENDPOINT = os.getenv("COSMOS_ENDPOINT")
   COSMOS_KEY = os.getenv("COSMOS_KEY")
   COSMOS_DATABASE_NAME = os.getenv("COSMOS_DATABASE_NAME")
   COSMOS_CONTAINER_NAME = os.getenv("COSMOS_CONTAINER_NAME")
   

2. •

   Create the Azure OpenAI client

   Use the official OpenAI Python SDK configured for Azure. This is the entry point for chat completions in your banking agent.

   from openai import AzureOpenAI
   
   client = AzureOpenAI(
       azure_endpoint=AZURE_OPENAI_ENDPOINT,
       api_key=AZURE_OPENAI_API_KEY,
       api_version="2024-06-01"
   )
   
   def generate_bank_reply(user_message: str) -> str:
       response = client.chat.completions.create(
           model=AZURE_OPENAI_DEPLOYMENT,
           messages=[
               {"role": "system", "content": "You are a banking assistant. Be precise, compliant, and concise."},
               {"role": "user", "content": user_message}
           ],
           temperature=0.2,
       )
       return response.choices[0].message.content
   

3. •

   Connect to Cosmos DB and create a container handle

   Cosmos DB works well for storing conversation history, customer profiles, risk flags, or retrieved context for later turns.

   from azure.cosmos import CosmosClient
   
   cosmos_client = CosmosClient(COSMOS_ENDPOINT, credential=COSMOS_KEY)
   database = cosmos_client.get_database_client(COSMOS_DATABASE_NAME)
   container = database.get_container_client(COSMOS_CONTAINER_NAME)
   
   def save_agent_memory(session_id: str, user_text: str, assistant_text: str):
       item = {
           "id": f"{session_id}:{len(user_text)}:{len(assistant_text)}",
           "sessionId": session_id,
           "userText": user_text,
           "assistantText": assistant_text,
       }
       container.upsert_item(item)
       return item
   

4. •

   Read context from Cosmos DB before calling Azure OpenAI

   This is the key integration pattern: fetch prior state from Cosmos DB, inject it into the prompt, then persist the response back into storage.

   def get_recent_session_messages(session_id: str):
       query = """
       SELECT TOP 5 c.userText, c.assistantText
       FROM c
       WHERE c.sessionId = @sessionId
       ORDER BY c._ts DESC
       """
       items = list(container.query_items(
           query=query,
           parameters=[{"name": "@sessionId", "value": session_id}],
           enable_cross_partition_query=True
       ))
       return items
   
   

def answer_with_memory(session_id: str, user_message: str) -> str: history = get_recent_session_messages(session_id)

   memory_block = "\n".join(
       [f"User: {h['userText']}\nAssistant: {h['assistantText']}" for h in history]
   ) or "No prior context."

   response = client.chat.completions.create(
       model=AZURE_OPENAI_DEPLOYMENT,
       messages=[
           {"role": "system", "content": "You are a banking support agent. Use only the provided context when relevant."},
           {"role": "system", "content": f"Conversation history:\n{memory_block}"},
           {"role": "user", "content": user_message},
       ],
       temperature=0.2,
   )

   answer = response.choices[0].message.content
   save_agent_memory(session_id, user_message, answer)
   return answer

5. **Wrap it in a single orchestration function**

This gives you one clean API surface for your startup app or backend service.

```python
def handle_banking_request(session_id: str, prompt: str) -> dict:
    reply = answer_with_memory(session_id=session_id, user_message=prompt)

    return {
        "sessionId": session_id,
        "response": reply,
        "source": "azure-openai + cosmosdb"
    }


if __name__ == "__main__":
 result = handle_banking_request(
     session_id="cust-1001",
     prompt="Summarize my last two support requests and draft a follow-up."
 )
 print(result["response"])

Testing the Integration

Run a simple end-to-end test that checks both generation and persistence.

test_session_id = "test-session-001"
test_prompt = "What is the status of my card replacement request?"

response_text = handle_banking_request(test_session_id, test_prompt)
print("Assistant:", response_text)

saved_items = list(container.query_items(
    query="SELECT * FROM c WHERE c.sessionId = @sessionId",
    parameters=[{"name": "@sessionId", "value": test_session_id}],
    enable_cross_partition_query=True
))

print("Saved records:", len(saved_items))

Expected output:

Assistant: Your card replacement request is being processed and should complete within 3 business days.
Saved records: 1

If you get a response back from Azure OpenAI and at least one document lands in Cosmos DB, your integration is working.

Real-World Use Cases

• •

  Banking support agent with memory

  • •Store conversation turns in Cosmos DB.
  • •Let Azure OpenAI summarize prior interactions and draft responses.
• •

  KYC or onboarding assistant

  • •Persist applicant metadata in Cosmos DB.
  • •Use Azure OpenAI to explain missing documents or generate next-step checklists.
• •

  Fraud triage copilot

  • •Save alerts, notes, and analyst actions in Cosmos DB.
  • •Use Azure OpenAI to turn raw alert data into analyst-ready summaries.

The pattern is straightforward: Cosmos DB holds durable state, Azure OpenAI produces the reasoning layer. For startups building banking agents, that split keeps your system auditable enough for production and flexible enough to ship fast.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.