LlamaIndex Tutorial (Python): chunking large documents for advanced developers

This tutorial shows you how to chunk large documents in LlamaIndex with Python, control chunk size and overlap, and inspect the resulting nodes before indexing. You need this when your source docs are too large for a single embedding pass, when retrieval quality drops on long PDFs, or when you want deterministic chunking for compliance-heavy workflows.

What You'll Need

• •Python 3.10+
• •llama-index
• •A document file to test with, such as a PDF or .txt
• •An LLM/embedding setup if you plan to query the index later
  • •For local testing, you can stop at parsing and chunk inspection
  • •For retrieval, configure an embedding model provider
• •Optional but useful:
  • •pymupdf for PDF loading
  • •python-dotenv for environment variables

Install the core package:

pip install llama-index pymupdf python-dotenv

Step-by-Step

1. •Start by loading a real document into LlamaIndex. For advanced chunking work, you want to inspect the raw text first so you know whether your splitter settings are sane.

from llama_index.core import SimpleDirectoryReader

docs = SimpleDirectoryReader(
    input_dir="./data",
    required_exts=[".txt"]
).load_data()

print(f"Loaded {len(docs)} document(s)")
print(docs[0].text[:500])

2. •Next, define your chunking strategy explicitly. The default settings are fine for demos, but production systems usually need tighter control over chunk size and overlap so retrieval doesn’t miss context at boundaries.

from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(
    chunk_size=512,
    chunk_overlap=64,
)

nodes = splitter.get_nodes_from_documents(docs)

print(f"Created {len(nodes)} chunks")
for i, node in enumerate(nodes[:3]):
    print(f"\nChunk {i + 1}")
    print(node.text[:400])

3. •If you need more control over how chunks are produced, use metadata-aware parsing and inspect node boundaries. This is useful when you want traceability back to the original file and line up chunks with business sections like policy clauses or account terms.

from llama_index.core import Document
from llama_index.core.node_parser import SentenceSplitter

with open("./data/sample.txt", "r", encoding="utf-8") as f:
    text = f.read()

doc = Document(
    text=text,
    metadata={"source": "sample.txt", "doc_type": "policy"}
)

splitter = SentenceSplitter(chunk_size=400, chunk_overlap=50)
nodes = splitter.get_nodes_from_documents([doc])

for node in nodes[:2]:
    print(node.metadata)
    print(node.text[:300])
    print("-" * 40)

4. •For long documents with mixed structure, compare different chunk sizes before committing to one setting. Smaller chunks improve precision; larger chunks preserve context; the right answer depends on your retrieval task.

from llama_index.core.node_parser import SentenceSplitter

settings = [
    {"chunk_size": 256, "chunk_overlap": 32},
    {"chunk_size": 512, "chunk_overlap": 64},
    {"chunk_size": 1024, "chunk_overlap": 128},
]

for cfg in settings:
    splitter = SentenceSplitter(**cfg)
    nodes = splitter.get_nodes_from_documents(docs)
    avg_len = sum(len(n.text) for n in nodes) / len(nodes)
    print(cfg, "chunks:", len(nodes), "avg_chars:", round(avg_len))

5. •Once you have a good splitter configuration, build an index from those chunks. This keeps the same chunking logic used during ingestion and makes it easy to query later with consistent retrieval behavior.

from llama_index.core import VectorStoreIndex
from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(chunk_size=512, chunk_overlap=64)
nodes = splitter.get_nodes_from_documents(docs)

index = VectorStoreIndex(nodes)
query_engine = index.as_query_engine()

response = query_engine.query("What does the document say about termination?")
print(response)

Testing It

Verify that the number of chunks changes when you adjust chunk_size; if it doesn’t, you’re probably not splitting the same document object you think you are. Check that each printed node contains coherent text instead of mid-sentence fragments or giant walls of text.

If you build an index, ask a question whose answer appears near a section boundary in the source file. That’s where overlap matters most: if retrieval fails there, increase overlap before increasing chunk size.

For compliance or audit use cases, confirm that each node’s metadata still points back to the original source file. If provenance is missing, fix that before moving on to embeddings or vector storage.

Next Steps

• •Try TokenTextSplitter instead of SentenceSplitter if your downstream model is token-budget constrained.
• •Add a custom preprocessing step for headers, footers, tables, and OCR noise before splitting.
• •Move from local inspection to persistent storage with a vector database like PostgreSQL/pgvector or Pinecone.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.