CrewAI Tutorial (Python): persisting agent state for advanced developers

This tutorial shows you how to persist CrewAI agent state in Python so your agents can resume work, keep memory across runs, and avoid losing context after process restarts. You need this when your workflow spans multiple tasks, long-running jobs, or production systems where you cannot rely on in-memory state.

What You'll Need

• •Python 3.10+
• •crewai
• •python-dotenv
• •An OpenAI API key set as OPENAI_API_KEY
• •A writable local directory for storing persisted state
• •Basic familiarity with CrewAI Agent, Task, and Crew

Step-by-Step

1. •Start with a clean project and install the dependencies.
   I’m using local file persistence here because it is easy to inspect and works well for development and internal tools.

pip install crewai python-dotenv

2. •Set up your environment variables and a small persistence folder.
   CrewAI does not magically persist state unless you design for it, so we’ll store agent memory data in a JSON file between runs.

import os
import json
from pathlib import Path
from dotenv import load_dotenv

load_dotenv()

STATE_DIR = Path("state")
STATE_DIR.mkdir(exist_ok=True)

STATE_FILE = STATE_DIR / "agent_state.json"
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

if not OPENAI_API_KEY:
    raise ValueError("OPENAI_API_KEY is required")

3. •Define a tiny state store that can load and save JSON safely.
   This gives you a durable place to keep agent context such as notes, decisions, or the last completed step.

from typing import Any, Dict

def load_state() -> Dict[str, Any]:
    if not STATE_FILE.exists():
        return {"notes": [], "last_result": None}
    with STATE_FILE.open("r", encoding="utf-8") as f:
        return json.load(f)

def save_state(state: Dict[str, Any]) -> None:
    with STATE_FILE.open("w", encoding="utf-8") as f:
        json.dump(state, f, indent=2)

4. •Build an agent that reads prior state before it works and writes new state after it finishes.
   The important pattern here is simple: inject persisted context into the task prompt, then update the store with the result once the run completes.

from crewai import Agent, Task, Crew, Process

state = load_state()

agent = Agent(
    role="Research Analyst",
    goal="Summarize findings while preserving working context",
    backstory="You maintain continuity across runs by using stored notes.",
    verbose=True,
)

task = Task(
    description=(
        "Use the previous notes below as context.\n\n"
        f"Previous notes: {state['notes']}\n"
        f"Last result: {state['last_result']}\n\n"
        "Write a concise status update for the next run."
    ),
    expected_output="A short status update with one recommendation.",
    agent=agent,
)

5. •Run the crew and persist the output immediately after execution.
   In production, this is where you would also persist metadata like timestamps, task IDs, or failure reasons.

crew = Crew(
    agents=[agent],
    tasks=[task],
    process=Process.sequential,
    verbose=True,
)

result = crew.kickoff()

state["notes"].append(str(result))
state["last_result"] = str(result)
save_state(state)

print(result)
print(f"State saved to: {STATE_FILE}")

6. •Add a second run path to prove persistence works across executions.
   On the next invocation, the agent will see prior notes in the prompt and continue from there instead of starting cold.

if __name__ == "__main__":
    current_state = load_state()
    print("Loaded state:")
    print(json.dumps(current_state, indent=2))

    crew_result = crew.kickoff()
    current_state["notes"].append(str(crew_result))
    current_state["last_result"] = str(crew_result)
    save_state(current_state)

Testing It

Run the script twice from the same directory. On the first run, state/agent_state.json should be created with initial output; on the second run, the agent prompt should include the prior notes and last result from disk. If you want stronger verification, delete only part of the JSON file and confirm your loader falls back cleanly when no file exists.

Watch for two things in the console: whether Loaded state: shows previous content and whether new output gets appended instead of replacing old data. That tells you your persistence layer is actually carrying state across runs rather than just printing a response.

Next Steps

• •Replace the JSON file with SQLite or Postgres if you need concurrent writes or auditability.
• •Persist structured fields like task_status, tool_calls, and human_approval_required instead of raw text blobs.
• •Wrap this pattern in a repository/service class so every CrewAI workflow in your codebase uses the same state contract.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.