LlamaIndex Tutorial (Python): adding tool use for intermediate developers

This tutorial shows how to add tool use to a LlamaIndex agent in Python so it can call external functions instead of only answering from indexed data. You need this when your assistant has to do real work like fetching live data, looking up account status, or running business logic that should not live inside the prompt.

What You'll Need

• •Python 3.10+
• •llama-index
• •llama-index-llms-openai
• •An OpenAI API key
• •Basic familiarity with LlamaIndex Settings, retrievers, and query engines
• •A terminal with pip installed

Install the packages first:

pip install llama-index llama-index-llms-openai

Set your API key:

export OPENAI_API_KEY="your-key-here"

Step-by-Step

1. •Start with a minimal LlamaIndex setup and define a small tool function. Keep the tool deterministic and narrow in scope; agents work better when tools do one thing well.

import os
from typing import Optional

from llama_index.core import Settings, VectorStoreIndex, Document
from llama_index.llms.openai import OpenAI

Settings.llm = OpenAI(model="gpt-4o-mini")

def get_policy_status(policy_id: str) -> str:
    mock_db = {
        "POL-1001": "Active",
        "POL-1002": "Lapsed",
        "POL-1003": "Pending underwriting",
    }
    return mock_db.get(policy_id, "Policy not found")

docs = [
    Document(text="Claims are reviewed within 2 business days."),
    Document(text="Premium payments are due on the 1st of each month."),
]

index = VectorStoreIndex.from_documents(docs)
query_engine = index.as_query_engine()

2. •Wrap the function as a LlamaIndex tool. This is the bridge between normal Python code and agent reasoning; once wrapped, the model can decide when to call it.

from llama_index.core.tools import FunctionTool

policy_tool = FunctionTool.from_defaults(
    fn=get_policy_status,
    name="get_policy_status",
    description="Look up the current status of an insurance policy by policy ID.",
)

3. •Build an agent that can use both the index and the tool. Here we combine document retrieval with action-taking so the model can answer policy questions and also fetch live status from the function.

from llama_index.core.agent import ReActAgent

tools = [policy_tool]

agent = ReActAgent.from_tools(
    tools=tools,
    llm=Settings.llm,
    verbose=True,
)

response_1 = agent.chat("What is the status of policy POL-1001?")
print(response_1)

response_2 = agent.chat("When are premium payments due?")
print(response_2)

4. •Add your index as a query tool so the agent can retrieve from documents too. This is the common production pattern: one set of tools for system actions, another for knowledge retrieval.

from llama_index.core.tools import QueryEngineTool, ToolMetadata

kb_tool = QueryEngineTool.from_defaults(
    query_engine=query_engine,
    metadata=ToolMetadata(
        name="knowledge_base",
        description="Answers questions about claims timing and premium payment rules.",
    ),
)

agent_with_kb = ReActAgent.from_tools(
    tools=[policy_tool, kb_tool],
    llm=Settings.llm,
    verbose=True,
)

print(agent_with_kb.chat("What is the status of policy POL-1002?"))
print(agent_with_kb.chat("How long does claim review take?"))

5. •Make the tool output more production-friendly by validating inputs before lookup. In real systems you should reject malformed IDs early instead of letting bad values propagate into downstream services.

import re

def get_policy_status_safe(policy_id: str) -> str:
    if not re.fullmatch(r"POL-\d{4}", policy_id):
        return "Invalid policy ID format"

    mock_db = {
        "POL-1001": "Active",
        "POL-1002": "Lapsed",
        "POL-1003": "Pending underwriting",
    }
    return mock_db.get(policy_id, "Policy not found")

safe_policy_tool = FunctionTool.from_defaults(
    fn=get_policy_status_safe,
    name="get_policy_status_safe",
    description="Look up policy status after validating that the policy ID matches POL-0000 format.",
)

Testing It

Run a few direct prompts and watch whether the agent chooses the right path. If you ask about POL-1001, it should call the policy tool; if you ask about claims timing, it should retrieve from the knowledge base instead.

Turn on verbose=True while testing so you can inspect tool selection and intermediate reasoning. That matters when an agent gives a wrong answer because you can tell whether it failed at retrieval, tool selection, or prompt interpretation.

Try malformed inputs too, like policy 12 or ABC123. Your safe tool should return a clean validation message rather than pretending it found a record.

Next Steps

• •Replace the mock dictionary with a real service call using requests or your internal SDK.
• •Add multiple domain tools: claims lookup, customer profile lookup, payment schedule lookup.
• •Move from ReActAgent to a workflow-based architecture once you need branching, retries, or human approval steps.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.