Haystack Tutorial (Python): adding tool use for beginners

This tutorial shows you how to add tool use to a Haystack pipeline in Python so an LLM can call external functions instead of guessing. You need this when your agent has to fetch live data, do calculations, or trigger internal business logic before answering.

What You'll Need

• •Python 3.10+
• •haystack-ai
• •An OpenAI API key
• •Internet access for installing packages
• •Basic familiarity with Haystack pipelines and components

Install the package:

pip install haystack-ai

Set your API key:

export OPENAI_API_KEY="your-api-key"

Step-by-Step

1. •Start with a simple tool function.

The tool should be a normal Python function with a clear name, inputs, and a deterministic output. For beginners, keep it small and testable.

from datetime import datetime

def get_bank_holiday(country_code: str) -> str:
    holidays = {
        "US": "2026-07-04",
        "GB": "2026-12-25",
        "ZA": "2026-12-25",
    }
    return holidays.get(country_code.upper(), "Unknown country code")

def get_current_utc_time() -> str:
    return datetime.utcnow().isoformat(timespec="seconds") + "Z"

print(get_bank_holiday("US"))
print(get_current_utc_time())

2. •Wrap the function as a Haystack tool.

Haystack tools are regular callables decorated with @tool. The docstring matters because the model uses it to decide when to call the tool and what arguments to pass.

from haystack.tools import tool

@tool
def get_bank_holiday(country_code: str) -> str:
    """
    Return the next major bank holiday date for a given country code.
    
    Args:
        country_code: ISO-like country code such as US, GB, or ZA.
    """
    holidays = {
        "US": "2026-07-04",
        "GB": "2026-12-25",
        "ZA": "2026-12-25",
    }
    return holidays.get(country_code.upper(), "Unknown country code")

3. •Create an OpenAI chat generator that can use tools.

The OpenAIChatGenerator is the LLM component that will decide whether to answer directly or call your tool first. Make sure the model you choose supports tool calling.

from haystack.components.generators.chat import OpenAIChatGenerator

generator = OpenAIChatGenerator(
    model="gpt-4o-mini",
    generation_kwargs={
        "temperature": 0,
    },
)

4. •Build a pipeline that passes messages and tools into the generator.

This is the core wiring. You send in chat messages plus the available tools, and Haystack handles the tool-call loop through the component output.

from haystack import Pipeline
from haystack.dataclasses import ChatMessage

pipe = Pipeline()
pipe.add_component("llm", generator)

messages = [
    ChatMessage.from_system(
        "You are a helpful assistant. Use tools when needed."
    ),
    ChatMessage.from_user(
        "What is the next bank holiday in GB?"
    ),
]

result = pipe.run(
    {
        "llm": {
            "messages": messages,
            "tools": [get_bank_holiday],
        }
    }
)

print(result["llm"]["replies"][0].text)

5. •Add a second tool so you can see selection behavior.

Once one tool works, add another one with a different purpose. This helps you verify that the model picks the right function based on the user request instead of always calling the same tool.

@tool
def calculate_vat(amount: float, rate: float) -> float:
    """
    Calculate VAT for a given amount.
    
    Args:
        amount: Base amount before tax.
        rate: VAT rate as a decimal, for example 0.15.
    """
    return round(amount * rate, 2)

messages = [
    ChatMessage.from_system("Use tools when useful."),
    ChatMessage.from_user("Calculate VAT on 1200 at 15%."),
]

result = pipe.run(
    {
        "llm": {
            "messages": messages,
            "tools": [get_bank_holiday, calculate_vat],
        }
    }
)

print(result["llm"]["replies"][0].text)

Testing It

Run the script and ask two different questions: one that needs get_bank_holiday and one that needs calculate_vat. If tool use is working, the model should call the relevant function and return an answer based on its output rather than inventing values.

If you want to confirm it is actually using tools, temporarily change one function’s return value to something obvious like "TOOL_CALLED". Then rerun the pipeline and check whether that exact value appears in the final response.

If you get an authentication error, check that OPENAI_API_KEY is set in your environment before starting Python. If you get a model error about tools not being supported, switch to a chat model that supports function calling.

Next Steps

• •Add more structured tools for real systems like customer lookup, policy status checks, or payment calculation
• •Learn how to connect Haystack tools to retrievers so your agent can search internal documents before answering
• •Add guardrails around tool inputs so your agent validates arguments before calling production services

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.