How to Fix 'state not updating when scaling' in CrewAI (Python)

When CrewAI says the state is not updating when scaling, it usually means your agents are running, but the data you expect to persist across tasks, retries, or parallel execution is being lost or overwritten. In practice, this shows up when you move from a single-agent flow to multiple agents, hierarchical crews, or async_execution=True.

The root cause is almost always one of these: mutable state passed incorrectly, state not returned from tasks, or parallel tasks writing to the same object without synchronization.

The Most Common Cause

The #1 cause is passing a plain Python dict around and assuming CrewAI will mutate it in place across tasks. That works in toy scripts, then breaks when you scale because each task may get its own copy, or your updates happen in a local variable that never gets returned.

Broken vs fixed pattern

# BROKEN
from crewai import Agent, Task, Crew

shared_state = {"approved": False, "risk_score": None}

risk_agent = Agent(
    role="Risk Analyst",
    goal="Assess risk",
    backstory="You evaluate loan applications."
)

task = Task(
    description="Update shared_state with risk score and approval.",
    agent=risk_agent,
    expected_output="A JSON update to shared_state"
)

crew = Crew(agents=[risk_agent], tasks=[task])

result = crew.kickoff(inputs={"shared_state": shared_state})
print(shared_state)  # often unchanged

# FIXED
from crewai import Agent, Task, Crew

risk_agent = Agent(
    role="Risk Analyst",
    goal="Assess risk",
    backstory="You evaluate loan applications."
)

task = Task(
    description=(
        "Return JSON with keys: approved (bool), risk_score (int), "
        "reason (string). Do not mention shared state."
    ),
    agent=risk_agent,
    expected_output='{"approved": true, "risk_score": 72, "reason": "..."}'
)

crew = Crew(agents=[risk_agent], tasks=[task])

result = crew.kickoff()
# Parse result and merge into your app state explicitly

The key difference: CrewAI is not your state store. Treat task output as an event, then apply it to your application state yourself.

If you need real state management, use:

• •a dataclass or Pydantic model
• •explicit return values from each task
• •a merge step in your orchestrator

Other Possible Causes

1) Parallel tasks are racing each other

When you use async_execution=True, two tasks can read the same old value and write conflicting updates. This often appears as “state not updating” when the real issue is last write wins.

task_1 = Task(
    description="Update the decision state",
    agent=agent_a,
    async_execution=True
)

task_2 = Task(
    description="Update the decision state",
    agent=agent_b,
    async_execution=True
)

Fix:

• •avoid writing directly to shared mutable objects
• •collect outputs separately
• •merge deterministically after both complete

2) You are relying on context incorrectly

CrewAI task context is for passing information forward, not for mutating upstream objects. If task B expects task A to modify a Python object in place, that assumption usually fails.

task_b = Task(
    description="Use the updated application state",
    agent=agent_b,
    context=[task_a]
)

Fix:

• •make task A return structured output
• •have task B consume that output explicitly in its prompt or input mapping

3) Your tool writes are not persisted

If an agent uses a tool that updates a database or cache but your code never commits the transaction, it looks like CrewAI failed. The issue is actually outside CrewAI.

class UpdateCaseTool(BaseTool):
    name = "update_case"

    def _run(self, case_id: str, status: str):
        db.case.status = status
        # missing db.commit()
        return {"ok": True}

Fix:

• •commit after writes
• •log tool inputs/outputs
• •verify persistence layer separately from the crew run

4) You’re hitting serialization limits

If you pass complex objects like SQLAlchemy sessions, open file handles, or custom classes through inputs, they may not serialize cleanly. Then downstream tasks receive incomplete data and appear to “not update.”

crew.kickoff(inputs={
    "session": db_session,   # bad idea
    "case": case_object      # often problematic
})

Fix:

• •pass primitives only: strings, numbers, lists, dicts of primitives
• •reconstruct complex objects inside your app layer

How to Debug It

1. •

   Print every task output

   • •Don’t guess.
   • •Log what each Task returns before any merge logic.
2. •

   Check whether you are mutating or returning

   • •If your code does state["x"] = ... inside an agent/tool path but never returns that value, assume it will be lost.
3. •

   Disable parallel execution

   • •Set async_execution=False temporarily.
   • •If the bug disappears, you have a race condition.
4. •

   Reduce inputs to primitives

   • •Replace custom objects with plain dicts.
   • •If the problem disappears, serialization was the issue.

A good debugging pattern looks like this:

result_a = crew_a.kickoff()
print("TASK A:", result_a)

result_b = crew_b.kickoff(inputs={"previous_result": result_a})
print("TASK B:", result_b)

If TASK A has the right data but TASK B doesn’t see it, the bug is in how you pass context forward.

Prevention

• •Use explicit outputs between tasks instead of shared mutable dictionaries.
• •Keep task inputs serializable: primitives and simple dicts only.
• •Treat parallel crews as read-only until the final merge step.
• •Add logging at every boundary: tool call, task output, merge step.

If you build CrewAI flows like distributed systems instead of local function calls, this error goes away fast. The rule is simple: agents produce outputs; your app owns state.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.