LangChain Tutorial (Python): adding observability for beginners

This tutorial shows you how to add basic observability to a LangChain Python app using LangSmith, so you can inspect prompts, model outputs, latency, and failures without guessing. You need this when your chain works locally but you want to see what happened in production, especially when debugging bad answers or tracking regressions.

What You'll Need

• •Python 3.10+
• •A LangChain project already set up
• •An OpenAI API key
• •A LangSmith account and API key
• •Installed packages:
  • •langchain
  • •langchain-openai
  • •langsmith
  • •python-dotenv

Install them with:

pip install langchain langchain-openai langsmith python-dotenv

Step-by-Step

1. •Set your environment variables first. LangSmith reads tracing settings from env vars, and this is the cleanest way to turn observability on without changing your application logic.

export OPENAI_API_KEY="your-openai-api-key"
export LANGCHAIN_TRACING_V2="true"
export LANGCHAIN_API_KEY="your-langsmith-api-key"
export LANGCHAIN_PROJECT="langchain-observability-demo"

2. •Create a simple .env file if you prefer local development over shell exports. This keeps secrets out of your code and makes it easier to run the same app across machines.

OPENAI_API_KEY=your-openai-api-key
LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=your-langsmith-api-key
LANGCHAIN_PROJECT=langchain-observability-demo

3. •Load those variables in Python and build a small chain. The important part is that LangChain will automatically emit traces once tracing is enabled, so you do not need special callback code for the basic setup.

from dotenv import load_dotenv
load_dotenv()

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a concise assistant."),
    ("user", "Explain {topic} in one sentence.")
])

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chain = prompt | llm

result = chain.invoke({"topic": "LangSmith observability"})
print(result.content)

4. •Add structured tracing metadata when you call the chain. This is where observability becomes useful in real systems, because you can tag requests by customer, environment, or feature flag and filter them later in LangSmith.

from dotenv import load_dotenv
load_dotenv()

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a concise assistant."),
    ("user", "Summarize this for a bank analyst: {text}")
])

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chain = prompt | llm

response = chain.invoke(
    {"text": "Customer disputes increased 12% after card replacement."},
    config={
        "tags": ["demo", "observability"],
        "metadata": {"team": "risk", "env": "dev"}
    }
)

print(response.content)

5. •Wrap the same pattern into a reusable function for your app. In production, this keeps tracing consistent across endpoints, jobs, and agents instead of relying on developers to remember manual setup.

from dotenv import load_dotenv
load_dotenv()

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You write short internal notes."),
    ("user", "Draft a note about: {incident}")
])

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chain = prompt | llm

def generate_note(incident: str) -> str:
    result = chain.invoke(
        {"incident": incident},
        config={"tags": ["notes"], "metadata": {"service": "ops-assistant"}}
    )
    return result.content

print(generate_note("Payment retries failing for Visa cards"))

6. •If you want deeper visibility later, start adding callbacks and custom events around tools or retrievers. For beginners, the trace from invoke() is enough; once you have that working, instrumenting more components is straightforward.

from dotenv import load_dotenv
load_dotenv()

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a support assistant."),
    ("user", "{question}")
])

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chain = prompt | llm

questions = [
    "What does a 500 error mean?",
    "How do I check retry logs?"
]

for q in questions:
    answer = chain.invoke(
        {"question": q},
        config={"tags": ["support"], "metadata": {"channel": "internal"}}
    )
    print(answer.content)

Testing It

Run the script once and confirm it returns model output in your terminal. Then open LangSmith and check that a new project named langchain-observability-demo contains traces for each invoke() call.

If you see no traces, check these first:

• •LANGCHAIN_TRACING_V2=true is set before running the script
• •LANGCHAIN_API_KEY is valid
• •LANGCHAIN_PROJECT matches the project name you expect
• •Your OpenAI key is valid and has access to the selected model

The trace should show the prompt messages, model response, runtime metadata, and any tags you passed in config. That gives you enough signal to debug prompt changes and compare runs across environments.

Next Steps

• •Add observability to tools and retrievers so you can trace full agent execution paths.
• •Learn how to use LangSmith datasets for regression testing prompts before deployment.
• •Add request IDs or customer IDs as metadata so traces map back to real application traffic.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.