LlamaIndex Tutorial (Python): deploying with Docker for beginners

This tutorial shows you how to package a small LlamaIndex Python app into a Docker container and run it locally. You need this when you want the same app behavior on your laptop, in CI, or in a server environment without “works on my machine” problems.

What You'll Need

• •Python 3.10 or newer
• •Docker Desktop or Docker Engine installed
• •An OpenAI API key set as an environment variable
• •A basic LlamaIndex install:
  • •llama-index
  • •llama-index-llms-openai
  • •llama-index-embeddings-openai
• •A working internet connection for pulling packages and calling the API

Step-by-Step

1. •Create a minimal project structure. Keep the app small first: one Python file, one requirements file, and one Dockerfile.

mkdir llamaindex-docker-demo
cd llamaindex-docker-demo
touch app.py requirements.txt Dockerfile .dockerignore

2. •Add the Python app. This example uses VectorStoreIndex with a simple in-memory document and an OpenAI-backed query engine. It’s enough to prove your container can run LlamaIndex end to end.

import os

from llama_index.core import Document, VectorStoreIndex
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

def main() -> None:
    api_key = os.environ["OPENAI_API_KEY"]

    llm = OpenAI(model="gpt-4o-mini", api_key=api_key)
    embed_model = OpenAIEmbedding(model="text-embedding-3-small", api_key=api_key)

    docs = [
        Document(text="LlamaIndex helps connect data sources to LLM applications."),
        Document(text="Docker packages code and dependencies into a portable container."),
    ]

    index = VectorStoreIndex.from_documents(docs, embed_model=embed_model)
    query_engine = index.as_query_engine(llm=llm)

    response = query_engine.query("What does Docker do?")
    print(response)

if __name__ == "__main__":
    main()

3. •Pin the dependencies. Don’t rely on implicit transitive installs if you want predictable builds.

llama-index==0.11.23
llama-index-llms-openai==0.2.16
llama-index-embeddings-openai==0.2.5

4. •Build the Docker image carefully. Use a slim Python base image, install dependencies first for better layer caching, then copy the app code.

FROM python:3.11-slim

WORKDIR /app

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

CMD ["python", "app.py"]

5. •Ignore local junk files so your image stays clean. This keeps caches, virtual environments, and secrets out of the build context.

__pycache__/
*.pyc
.env
venv/
.envrc
.git/
.DS_Store

6. •Build and run the container with your API key passed at runtime. That keeps secrets out of the image and makes it easier to rotate credentials later.

docker build -t llamaindex-docker-demo .

docker run --rm \
  -e OPENAI_API_KEY="$OPENAI_API_KEY" \
  llamaindex-docker-demo

Testing It

If everything is wired correctly, the container should print a natural-language answer from the query engine instead of crashing on imports or missing credentials. If you see an authentication error, check that OPENAI_API_KEY is available in your shell before running docker run. If you see dependency errors, verify that the package versions in requirements.txt match the imports in app.py.

A good sanity check is to change the query text in app.py and rebuild the image:

• •Update "What does Docker do?" to "What is LlamaIndex?"
• •Rebuild with docker build -t llamaindex-docker-demo .
• •Run again and confirm the output changes accordingly

Next Steps

• •Add a .env workflow with docker run --env-file .env ... for local development.
• •Replace the in-memory documents with a real data source like PDFs, Notion pages, or S3 files.
• •Move from a single-file script to a FastAPI service so your LlamaIndex app can serve requests over HTTP

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.