LangChain Tutorial (Python): adding audit logs for beginners

This tutorial shows you how to add audit logs to a LangChain Python app so every prompt, response, tool call, and error is recorded. You need this when you want traceability for compliance, debugging, incident review, or just to understand what your agent actually did in production.

What You'll Need

• •Python 3.10+
• •A virtual environment
• •langchain
• •langchain-openai
• •openai API key
• •Basic familiarity with LLMChain or chat model calls
• •A writable local file for logs

Install the packages:

pip install langchain langchain-openai openai

Set your API key:

export OPENAI_API_KEY="your-key-here"

Step-by-Step

1. •Start with a small LangChain app that calls a chat model. Keep the chain simple so you can see exactly where to insert logging without mixing in other moving parts.

import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

messages = [
    HumanMessage(content="Write one sentence explaining what audit logs are.")
]

response = llm.invoke(messages)
print(response.content)

2. •Create a logger that writes structured JSON lines to disk. JSONL is easy to grep, ship to a log pipeline, or parse later when you need to reconstruct an execution.

import json
import logging
from datetime import datetime, timezone

logger = logging.getLogger("audit")
logger.setLevel(logging.INFO)

handler = logging.FileHandler("audit.log")
handler.setFormatter(logging.Formatter("%(message)s"))
logger.addHandler(handler)

def audit_event(event_type: str, payload: dict) -> None:
    record = {
        "ts": datetime.now(timezone.utc).isoformat(),
        "event_type": event_type,
        "payload": payload,
    }
    logger.info(json.dumps(record))

3. •Wrap the model call so you log the input before execution and the output after execution. This is the basic pattern you want for production: capture enough context to debug, but avoid dumping secrets or raw customer data.

import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

prompt_text = "Write one sentence explaining what audit logs are."

audit_event("llm_request", {
    "model": "gpt-4o-mini",
    "temperature": 0,
    "input": prompt_text,
})

try:
    response = llm.invoke([HumanMessage(content=prompt_text)])
    audit_event("llm_response", {
        "model": "gpt-4o-mini",
        "output": response.content,
    })
    print(response.content)
except Exception as exc:
    audit_event("llm_error", {
        "model": "gpt-4o-mini",
        "error": type(exc).__name__,
        "message": str(exc),
    })
    raise

4. •Add tool-level auditing if your agent uses tools. In real systems, tool calls matter more than model text because they show side effects like database reads, ticket creation, or policy lookups.

from langchain_core.tools import tool

@tool
def lookup_policy(policy_id: str) -> str:
    """Mock policy lookup."""
    audit_event("tool_call", {"tool": "lookup_policy", "policy_id": policy_id})
    result = f"Policy {policy_id}: active"
    audit_event("tool_result", {"tool": "lookup_policy", "result": result})
    return result

print(lookup_policy.invoke("POL-12345"))

5. •If you want cleaner reuse, put the audit wrapper around a runnable function instead of scattering audit_event(...) calls everywhere. This keeps your chain code readable while still giving you consistent logs for every request.

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

def audited_chat(prompt_text: str) -> str:
    audit_event("llm_request", {"input": prompt_text})
    try:
        response = llm.invoke([HumanMessage(content=prompt_text)])
        output = response.content
        audit_event("llm_response", {"output": output})
        return output
    except Exception as exc:
        audit_event("llm_error", {
            "error": type(exc).__name__,
            "message": str(exc),
        })
        raise

print(audited_chat("Give me a short definition of an audit trail."))

Testing It

Run the script once and check that audit.log is created in the same directory. You should see one JSON object per line for the request and response events.

Then trigger an error by setting an invalid API key or disconnecting network access and confirm that an llm_error entry appears. That tells you your failure path is logged, not just the happy path.

If your app uses tools, call the tool function and verify both tool_call and tool_result entries are written. In production, this is the minimum bar for tracing what happened during an agent run.

Next Steps

• •Add request IDs and user IDs to every event so you can trace a single conversation end-to-end.
• •Ship JSONL logs to a central store like CloudWatch, ELK, or Datadog instead of keeping them only on disk.
• •Learn how to use LangChain callbacks so auditing happens automatically across chains, tools, and retrievers.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.