How to Integrate AutoGen for fintech with Docker for RAG

Combining AutoGen for fintech with Docker gives you a clean way to run retrieval-augmented agent workflows in isolated, reproducible environments. In practice, that means your finance assistant can query policy docs, product specs, or compliance notes from a containerized vector store without polluting the host machine or breaking across environments.

Prerequisites

• •Python 3.10+
• •Docker Desktop or Docker Engine installed and running
• •pip access to install Python packages
• •An AutoGen for fintech package installed in your environment
• •A working LLM API key configured for your AutoGen agents
• •A local folder with documents you want indexed for RAG
• •Basic familiarity with Python async code and Docker containers

Integration Steps

1. •

   Install the Python dependencies

   You need AutoGen, the Docker SDK, and a RAG stack. I use autogen-agentchat for agent orchestration and docker for container control.

   pip install autogen-agentchat docker chromadb sentence-transformers
   

2. •

   Start a Dockerized vector store for retrieval

   For fintech RAG, keep your document index in a container so it can be recreated consistently in dev, staging, and CI.

   import docker
   
   client = docker.from_env()
   
   container = client.containers.run(
       image="chromadb/chroma:latest",
       name="fintech-rag-chroma",
       detach=True,
       ports={"8000/tcp": 8000},
       environment={
           "CHROMA_SERVER_HOST": "0.0.0.0",
           "CHROMA_SERVER_HTTP_PORT": "8000",
       },
       restart_policy={"Name": "unless-stopped"},
   )
   
   print(f"Started container: {container.name}")
   

   If the container already exists, inspect it instead of creating a duplicate.

   import docker
   
   client = docker.from_env()
   try:
       container = client.containers.get("fintech-rag-chroma")
       if container.status != "running":
           container.start()
       print(container.status)
   except docker.errors.NotFound:
       print("Container not found")
   

3. •

   Create the retriever and connect it to AutoGen

   The pattern is simple: Docker hosts your vector DB, Python loads documents into it, and AutoGen calls the retriever when answering questions.

   from autogen_agentchat.agents import AssistantAgent
   from autogen_agentchat.messages import TextMessage
   from autogen_ext.models.openai import OpenAIChatCompletionClient
   
   model_client = OpenAIChatCompletionClient(
       model="gpt-4o-mini",
       api_key="YOUR_OPENAI_API_KEY",
   )
   
   assistant = AssistantAgent(
       name="fintech_assistant",
       model_client=model_client,
       system_message=(
           "You are a fintech support agent. "
           "Answer only from retrieved context when possible."
       ),
   )
   

4. •

   Index documents and expose retrieval as a tool

   Use Chroma as the backing store and wrap retrieval in a function AutoGen can call.

   import chromadb
   from chromadb.utils.embedding_functions import SentenceTransformerEmbeddingFunction
   
   chroma_client = chromadb.HttpClient(host="localhost", port=8000)
   
   embedding_fn = SentenceTransformerEmbeddingFunction(
       model_name="all-MiniLM-L6-v2"
   )
   
   collection = chroma_client.get_or_create_collection(
       name="fintech_docs",
       embedding_function=embedding_fn,
   )
   
   docs = [
       {
           "id": "policy_001",
           "text": "KYC must be completed before account activation.",
           "metadata": {"source": "onboarding_policy.pdf"},
       },
       {
           "id": "policy_002",
           "text": "Suspicious activity must be escalated within 24 hours.",
           "metadata": {"source": "aml_policy.pdf"},
       },
   ]
   
   collection.add(
       ids=[d["id"] for d in docs],
       documents=[d["text"] for d in docs],
       metadatas=[d["metadata"] for d in docs],
   )
   
   def retrieve_fintech_context(query: str) -> str:
       results = collection.query(query_texts=[query], n_results=3)
       chunks = results["documents"][0]
       sources = results["metadatas"][0]
       return "\n".join(
           f"- {chunk} (source: {meta['source']})"
           for chunk, meta in zip(chunks, sources)
       )
   

5. •

   Wire retrieval into the agent loop

   Call the retriever before generating the answer, then pass retrieved context into the assistant message.

    import asyncio
   
    async def answer_with_rag(question: str):
        context = retrieve_fintech_context(question)
   
        prompt = (
            f"Use this context:\n{context}\n\n"
            f"Question: {question}\n"
            f"Return a concise compliance-safe answer."
        )
   
        response = await assistant.on_messages(
            [TextMessage(content=prompt, source="user")],
            cancellation_token=None,
        )
        return response.chat_message.content
   
    if __name__ == "__main__":
        result = asyncio.run(answer_with_rag("When should KYC be completed?"))
        print(result)
   

Testing the Integration

Run a direct smoke test against both Docker and AutoGen. This verifies that the container is reachable, retrieval works, and the agent can generate an answer from indexed content.

import docker
import asyncio

client = docker.from_env()
container = client.containers.get("fintech-rag-chroma")
print(f"Container running: {container.status == 'running'}")

async def test():
    answer = await answer_with_rag("What is required before account activation?")
    print(answer)

asyncio.run(test())

Expected output:

Container running: True
KYC must be completed before account activation.

If you get a generic model answer instead of a grounded one, check these first:

• •The Chroma container is actually running on port 8000
• •Your collection contains documents before querying
• •Your embedding model matches between insert and query paths
• •The assistant prompt explicitly tells the model to use retrieved context

Real-World Use Cases

• •Compliance Q&A bot that answers policy questions from AML, KYC, PCI-DSS, or internal controls stored in Dockerized indexes.
• •Product support agent that retrieves pricing rules, eligibility criteria, and onboarding steps from versioned documentation.
• •Analyst copilot that summarizes procedures and generates draft responses using controlled context from finance knowledge bases.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.