The problem is, our chunking strategy for RAG is naive. What's the best library or framework for advanced text chunking?

Moving past a naive character-count splitter is one of the highest-ROI upgrades you can make to a RAG pipeline . Instead of a single standalone "magic" library, the ecosystem relies on a few powerhouse frameworks that specialize in structure-aware, recursive, and semantic chunking.

The top libraries and frameworks for advanced text chunking break down as follows:

• LangChain / LangChain Text Splitters : The industry standard for flexible text splitting. Beyond the basic RecursiveCharacterTextSplitter (which intelligently steps down through paragraphs, lines, and spaces), it offers specialized tools like MarkdownHeaderTextSplitter and HTMLHeaderTextSplitter to preserve document hierarchy, as well as experimental semantic chunking modules.
• LlamaIndex : Excellent if you want production-grade data ingestion pipelines. LlamaIndex offers advanced node parsers (their term for chunking) including SentenceSplitter , hierarchical node parsing (linking parent chunks to smaller child chunks for precise retrieval), and semantic-breakpoint chunking.
• Chonkie : A newer, lightweight, lightning-fast Python library built explicitly for RAG chunking. It strips out the heavy framework bloat of LangChain/LlamaIndex and provides clean, token-aware, semantic, and recursive chunkers designed for optimal retrieval performance.
• Unstructured : If your bottleneck is parsing the raw files (complex PDFs with multi-column layouts, embedded tables, and charts) before you even chunk them, Unstructured provides robust preprocessing and element-based partitioning that respects document semantics natively.
• Ragflow : An end-to-end open-source RAG engine that takes deep control over document parsing and intelligent layout-based chunking, heavily optimized for messy enterprise reports, tables, and scanned PDFs.

Which strategy should you adopt?

Don’t jump straight to expensive LLM-based ("agentic") or heavy semantic chunking unless your evaluation metrics explicitly demand it. Current benchmarks show that a well-tuned Recursive Character Splitter (targeting ~400–512 tokens with 10–20% overlap) outperforms or matches complex semantic methods on most standard data at a fraction of the computational cost.

To help narrow down the best tool or configuration for your stack:

• What types of documents are you primarily processing (e.g., dense PDFs, codebases, Markdown, messy Word docs)?
• Are you already locked into a framework like LangChain or LlamaIndex , or looking for a lightweight independent library?

If your current RAG chunking is basically “split every N tokens with overlap,” I’d move to a document-aware + semantic + hierarchical strategy rather than just swapping in a fancier fixed-size splitter.

My top pick: LlamaIndex

LlamaIndex is probably the best fit if chunking itself is the problem. Its node-parser system supports:

• Semantic chunking — detects topic shifts using embedding similarity.
• Hierarchical chunking — creates parent/child chunks at multiple granularities, then can retrieve a small child while supplying its larger parent context.
• Sentence-window chunking — retrieves a small unit but retains surrounding sentences as context.
• Structure-aware parsers for Markdown, HTML, code, etc.

The hierarchical approach is particularly interesting for RAG because it addresses a fundamental problem with naive chunking: the best retrieval unit isn't necessarily the best generation context.

For example:

Document
└── Section (2048 tokens)
    ├── Subsection (512)
    │   ├── Chunk (128)
    │   ├── Chunk (128)
    │   └── Chunk (128)
    └── Subsection (512)

You embed/retrieve the small chunks, but when several related children match, you can merge upward and give the LLM the larger context. LlamaIndex explicitly supports this pattern with HierarchicalNodeParser + AutoMergingRetriever.

If you already use LangChain

LangChain is perfectly capable. Its SemanticChunker uses embeddings to find semantic breakpoints rather than blindly cutting at a character/token boundary. LangChain also has structure-aware splitters for HTML/Markdown.

I'd choose it if you're already heavily invested in LangChain rather than introducing LlamaIndex solely for chunking.

If you want a dedicated chunking library

I'd also look at Chonkie. It's more narrowly focused on chunking and has semantic, code-aware and other specialized chunking approaches.

And for messy enterprise documents—PDFs with tables, headers, lists, DOCX, HTML, etc.—Unstructured is particularly useful because the important problem may actually be document parsing/layout preservation before chunking, not the splitter itself.

What I'd build

Rather than:

PDF → text → 500-token chunks → embeddings → vector DB

I'd do:

PDF/HTML/MD
    ↓
structure/layout extraction
    ↓
preserve headings + metadata + tables
    ↓
semantic/structural boundaries
    ↓
hierarchical chunks
    ↓
embeddings
    ↓
hybrid retrieval + reranking
    ↓
expand retrieved chunks to useful context

And importantly, don't assume semantic chunking is universally better. Recent work suggests that adaptive/document-specific chunking can outperform a single strategy across heterogeneous corpora.

My ranking

If you tell me what your corpus looks like (PDFs, Markdown docs, code, support tickets, legal docs, etc.) and what you're using for the vector DB/embeddings, I can recommend a specific chunking architecture and parameters rather than just a library.

Moving past naive character-counting is the single best upgrade you can make to a RAG pipeline . Instead of a standalone "magic" library, advanced text chunking is usually handled by comprehensive orchestration frameworks or specialized token/semantic splitters.

The top libraries and frameworks for advanced text chunking break down as follows:

• 1. LangChain / LangChain Text Splitters : The industry standard for structural and recursive chunking. It provides RecursiveCharacterTextSplitter (the gold standard default that gracefully steps down from paragraphs to sentences to words), language-specific code splitters, and MarkdownHeaderTextSplitter / HTMLHeaderTextSplitter to keep hierarchical context intact.
• 1. LlamaIndex : Exceptional if your documents have complex hierarchies or you need node-parsing strategies. LlamaIndex shines with SentenceSplitter , hierarchical node parsing (parent-child chunking where you retrieve a small chunk for precision but feed a larger parent chunk to the LLM), and semantic-edge parsing.
• 1. Chonkie : A newer, lightweight, lightning-fast python library built purely for RAG chunking. It cuts out the massive dependency bloat of LangChain/LlamaIndex and provides clean semantic, recursive, and word-based chunkers optimized for token limits.
• 1. semchunk : A fast, dedicated semantic text chunker that splits text into chunks of a target size while respecting semantic boundaries (using tokenizers rather than expensive LLM embeddings for every single line), making it cheap and efficient.

Which strategy should you actually use?

• Structural/Layout-aware splitting : If you have Markdown, HTML, or code, use structural splitters (LangChain Documentation ) to split strictly by headers rather than arbitrary sizes.
• Parent-Child / Hierarchical Chunking : Index small chunks (~128–256 tokens) for high-precision vector search, but link them to their parent chunks (~1024 tokens) so the LLM gets full paragraph/section context.
• Semantic Chunking : Good in theory for tracking topic shifts via embedding distance drops, but watch out for compute costs on massive enterprise corpora.

If you tell me what kind of documents you are processing (PDFs, legal contracts, codebases, or messy web scrapes) and your current tech stack , I can recommend the exact snippet or strategy to fix your pipeline.

If your problem is specifically “naive fixed-size/recursive chunks are hurting RAG retrieval,” I’d look at Chonkie first.

My ranking

1. github.com — best dedicated chunking library

• Purpose-built for RAG rather than being a general orchestration framework.
• Supports recursive, semantic, late, neural, code, table, and LLM/agentic chunking.
• Particularly interesting are: - SemanticChunker — finds boundaries from embedding similarity.
• LateChunker — gives chunks richer document-level context.
• NeuralChunker — ML-based boundary detection.
• SlumberChunker — uses an LLM to determine chunk boundaries.
• Lightweight enough that you can use it without adopting a whole RAG framework.

2. llamaindex.ai — best if you want chunking + retrieval

• Its SemanticSplitterNodeParser creates chunks from semantically related sentences rather than arbitrary token boundaries.
• More importantly, its HierarchicalNodeParser + AutoMergingRetriever lets you retrieve small precise nodes and then expand them to larger parent context when appropriate.
• That's often more useful than simply finding a “better chunk size.”

3. huggingface.co — worth experimenting with

• A newer approach that combines semantic chunking with an AI-based segmentation model.
• Its published benchmark reports improvements over recursive and fixed-size chunking on a legal RAG QA task, although I'd treat those results as vendor/community-reported rather than universal evidence.

What I'd actually build

I wouldn't jump directly from:

512 tokens + 50 overlap to:

LLM decides every chunk A better production pipeline is usually:

Document
   ↓
structure-aware parsing
   ↓
semantic/recursive chunking
   ↓
metadata enrichment
   ↓
embedding
   ↓
hybrid retrieval
   ↓
reranking
   ↓
context expansion

For example, with Chonkie, I'd start with:

from chonkie import SemanticChunker

chunker = SemanticChunker(
    chunk_size=512,
    similarity_threshold=0.75,
)

chunks = chunker.chunk(document)

Then benchmark that against your existing recursive splitter.

For long, structured documents, I'd also test hierarchical retrieval rather than simply making chunks larger. LlamaIndex's hierarchy is designed specifically around the idea that a small retrieved passage can be promoted to its parent context when necessary.

One important point

Better chunking isn't necessarily smaller or more semantic chunks. Your optimal strategy depends heavily on the corpus:

• Markdown/docs: recursive + heading-aware
• Books/research: hierarchical + semantic
• Legal/financial: structure-aware + semantic/hierarchical
• Code: AST-aware chunking
• Messy prose: semantic/neural
• Very long documents: late chunking or hierarchical retrieval
• Small, highly specific corpus: LLM/agentic chunking can be worth testing

Chonkie is unusually attractive because it lets you run these experiments within one library rather than implementing each strategy yourself.

If you tell me what kind of documents you're indexing (PDFs, Markdown, web pages, code, support tickets, legal docs, etc.) and what embedding model/vector DB you're using, I can recommend a concrete chunking + retrieval architecture and starting parameters.

Document
   ↓
structure-aware parsing
   ↓
semantic/recursive chunking
   ↓
metadata enrichment
   ↓
embedding
   ↓
hybrid retrieval
   ↓
reranking
   ↓
context expansion

For example, with Chonkie, I'd start with:

from chonkie import SemanticChunker

chunker = SemanticChunker(
    chunk_size=512,
    similarity_threshold=0.75,
)

chunks = chunker.chunk(document)

Then benchmark that against your existing recursive splitter.

For long, structured documents, I'd also test hierarchical retrieval rather than simply making chunks larger. LlamaIndex's hierarchy is designed specifically around the idea that a small retrieved passage can be promoted to its parent context when necessary.

One important point

Better chunking isn't necessarily smaller or more semantic chunks. Your optimal strategy depends heavily on the corpus:

The best and most widely adopted framework for advanced text chunking in RAG pipelines is LangChain (along with its experimental modules), paired with document parsers like Docling or data prep tools like Firecrawl.

Moving away from naive character splitting requires adopting smart, structure-aware, or semantic strategies.

Advanced Chunking Frameworks & Libraries

• LangChain Text Splitters: Provides out-of-the-box implementations for Recursive Character splitting, Markdown/HTML header-based splitting, and semantic chunking via langchain_experimental.
• Semantic Chunkers Library: A specialized tool focused entirely on embedding-based semantic boundaries (statistical, consecutive, and cumulative methods).
• Docling: Excellent for document layout analysis, converting complex PDFs and structured documents into clean hierarchies before any chunking occurs.
• Firecrawl: Specializes in web data collection and clean markdown extraction to feed structured inputs directly into advanced splitters.

Top Advanced Chunking Strategies Compared

If you tell me what type of documents you are processing (e.g., messy PDFs, codebases, or legal contracts) and your current tech stack , I can recommend a specific implementation setup.

If you're moving beyond naive RecursiveCharacterTextSplitter-style chunking, my pick would be Chonkie for a dedicated chunking library, or LlamaIndex if you want chunking as part of a broader RAG framework.

My ranking

1. Chonkie — my first choice if chunking itself is the problem.
It specifically supports semantic, late, neural, code, recursive, and LLM-based chunkers. That's unusually broad for a lightweight library.

2. LlamaIndex — my choice for a serious production RAG system.
The interesting part isn't just its splitters. LlamaIndex supports approaches where the thing you embed/retrieve isn't necessarily the same thing you give the LLM. For example, you can embed a sentence but retrieve a surrounding sentence window, or retrieve via document summaries and then drill down to chunks.

3. Unstructured — particularly important if your corpus is PDFs/docs.
Instead of blindly splitting every N tokens, it first identifies document elements such as titles, paragraphs, tables, etc., then combines those elements into appropriately sized chunks. That's much better for reports, manuals, presentations, and other structured documents.

But there's an important catch

I wouldn't simply replace your naive splitter with "semantic chunking" and call it done.

A stronger modern architecture is something like:

document → structure/layout parsing → semantic/structural chunks → contextual enrichment → embedding → retrieval → neighboring/context expansion

Two techniques I'd especially investigate:

• Late chunking: embed a longer document first and derive chunk representations from the contextualized token embeddings. Chonkie has a LateChunker.
• Contextual retrieval: add document-specific context to each chunk before embedding. Anthropic reports substantially lower retrieval failure rates with this approach.

So if you told me "I have an existing RAG pipeline and retrieval quality is bad because chunks are terrible," I'd start with Chonkie + contextual/late chunking, while using Unstructured upstream if the corpus contains complex PDFs/documents.

If you tell me what your corpus looks like (PDFs, Markdown, web pages, code, support tickets, etc.) and what vector DB you're using, I can recommend a specific chunking architecture rather than just a library.

Moving away from a naive character-splitter is one of the highest-ROI upgrades you can make to a RAG pipeline . Rather than relying on massive all-in-one frameworks just for text splitting, the ecosystem has shifted toward dedicated, lightweight chunking libraries or specialized data-centric parsers.

Top Libraries & Frameworks for Advanced Chunking

• Chonkie : A lightweight, dedicated Python library built specifically for high-performance text chunking without the massive dependency tree or overhead of a full orchestrator. It natively supports token-aware windows, recursive hierarchy, and semantic splitting, making it easy to plug into any vector database or pipeline.
• Semchunk : A ultra-fast, text-only semantic splitter optimized for raw speed (up to 85% faster than alternative semantic splitters). It features built-in integrations for tiktoken and HuggingFace tokenizers, ensuring precise token boundaries.
• Chunklet-py : A modern, language-agnostic, and multilingual library (supporting 36+ languages) that implements clause-level overlaps and specialized splitters for text, documentation, and code. It handles semantic continuity naturally at natural clause boundaries rather than arbitrary character indexes.
• LlamaIndex : Best if you want a complete data-ingestion framework rather than just a function. Its advanced NodeParsers and hierarchical node structures natively support parent-child chunking (retrieving small precise chunks while feeding larger parent context blocks to the LLM).
• LangChain (Text Splitters) : The classic choice for modular workflows. Its RecursiveCharacterTextSplitter, MarkdownHeaderTextSplitter , and code-aware splitters are great for structural baseline parsing, though it can feel heavy if you only need advanced semantic algorithms.

Which Advanced Strategy Should You Implement?

1. Structure-Aware (The Practical Production Standard): Instead of cutting by raw size first, split by document elements (Markdown headers, HTML tags, or code AST via tree-sitter ). Keep tables and code blocks intact.
2. Hierarchical / Parent-Child Splitting: Index small chunks (~ 128 - 256 tokens) for maximum embedding precision, but map them to a parent chunk (~ 1024 tokens). When a match occurs, feed the larger parent block to the LLM to prevent context starvation.
3. Semantic Splitting (For Long-Form Prose): Use an embedding model or a cross-encoder to detect drastic drops in similarity between consecutive sentences, placing chunk boundaries only where the topic actually shifts.

If you can share what kind of documents you're processing (PDFs, messy HTML scrapes, legal contracts, codebases) and your current chunk size/overlap settings , I can recommend the ideal strategy and exact library configuration for your stack.

If your problem is specifically “fixed-size/recursive chunks are hurting retrieval quality,” I’d look at LlamaIndex first, with Unstructured alongside it if your source documents are messy PDFs/HTML/Office files.

My shortlist

My recommendation: use LlamaIndex's node-parsing machinery if you want to experiment with chunking strategies without rebuilding your RAG stack. Its semantic splitter uses embeddings to identify semantic breakpoints rather than blindly enforcing a token/character boundary.

If your documents are things like 10-Ks, manuals, policies, PDFs, slide decks, or reports, I'd actually put Unstructured before the chunker. It first identifies document elements such as titles, paragraphs, lists, and tables, then can chunk while preserving section boundaries. Its by_title strategy explicitly prevents a chunk from crossing section boundaries.

The bigger point: don't jump straight to “semantic chunks”

A good production setup is often:

document parsing → structural chunking → semantic refinement → parent/child retrieval → reranking

rather than:

PDF → 512-token chunks → embeddings

For example:

PDF
 ↓
layout/structure parser
 ↓
sections + paragraphs + tables
 ↓
semantic boundaries within sections
 ↓
small retrieval chunks
 ↓
retrieve parent section/context
 ↓
reranker
 ↓
LLM

There is also interesting newer work on AI-assisted chunking: semchunk's 2026 AI-chunking release reports better RAG correctness than its embedding-based mode and several conventional chunkers on a legal RAG benchmark. That's promising, although I'd treat those numbers as a benchmark result rather than proof that it will win on your corpus.

What I'd choose

• Mostly clean Markdown/text: LlamaIndex semantic/hierarchical chunking.
• Messy enterprise PDFs: Unstructured → LlamaIndex.
• Tables + text: Unstructured + separate table handling.
• Long documents where context gets lost: investigate parent-child or late chunking, not just smaller chunks.
• You already have LangChain: its splitters are perfectly usable; I wouldn't migrate frameworks solely for chunking.
• You want to push retrieval quality: benchmark semantic, hierarchical/parent-child, and late-chunking approaches against your actual queries.

One particularly important insight: chunking and retrieval strategy are coupled. A 200-token chunk can be excellent for vector retrieval while being terrible as the context given to the LLM. That's why parent-child retrieval and contextual/late chunking can outperform simply searching for the “perfect” chunk size.

If you tell me what your documents look like (PDFs, Markdown, code, legal docs, support tickets, etc.) and what vector DB/embedding model you're using, I can recommend a concrete chunking architecture and libraries rather than just a framework shortlist.

If you're moving beyond naive chunk_size=500, overlap=50, my pick depends on what your corpus looks like:

My shortlist

I'd start with LlamaIndex if you're building a serious RAG system. Its SemanticSplitterNodeParser groups semantically related sentences and uses embedding similarity to identify natural boundaries; it also supports metadata and previous/next relationships.

The more interesting architecture, though, is structure-aware → semantic, rather than simply "semantic chunking everywhere":

PDF / HTML / Markdown
        ↓
document parsing
        ↓
preserve hierarchy
(title → section → subsection)
        ↓
split at structural boundaries
        ↓
semantic merge/split within sections
        ↓
add contextual metadata
        ↓
embed
        ↓
retrieve + rerank

That tends to preserve things like "3.2 Authentication → OAuth → Token expiration" instead of turning them into anonymous text fragments.

One important caveat

Don't assume semantic chunking automatically improves RAG. Recent research has found that its extra embedding/computation cost doesn't consistently translate into better retrieval, while newer work suggests adaptive, document-aware chunking can outperform a single strategy applied to everything.

So if I were implementing this today:

1. LlamaIndex → if you want the most capable general solution
2. Chonkie → if you want a lightweight, chunking-focused library
3. Unstructured + LlamaIndex/your own retrieval → if PDFs and complex enterprise documents are the problem
4. LangChain → if you're already deeply invested in LangChain

And I'd benchmark 3–4 chunking strategies against your actual queries rather than picking one based on theory. Chunking is highly corpus-dependent.

If you tell me what you're chunking (e.g. PDFs, technical docs, code, legal docs, Markdown, support tickets) and your stack (Python/TypeScript + vector DB), I can recommend a concrete chunking pipeline and settings.

Moving away from naive character-counting splitters is one of the highest-leverage upgrades you can make to a RAG pipeline. Bad chunking breaks context mid-sentence, severs tables , and creates orphan snippets that even a top-tier embedding model or reranker cannot save.

The best tools and frameworks for advanced text chunking depend on whether you want a lightweight dedicated utility library or a full-scale orchestration framework.

Top Frameworks & Libraries for Advanced Chunking

1. Chonkie (Best Dedicated Lightweight Library)

• What it is: A fast, focused Python library designed solely for text chunking without the massive dependency overhead of heavy LLM frameworks.
• Why it’s advanced: It natively supports Semantic Chunker (splitting based on embedding similarity drops between sentences) and token-aware boundaries out of the box, making it exceptionally clean to drop into an existing pipeline.
• Best for: Projects where you already handle orchestration/vector storage yourself and just need a state-of-the-art, high-performance splitting engine.

2. LlamaIndex (Best for Data-Centric RAG Ingestion)

• What it is: A comprehensive data framework built specifically for RAG ingestion, indexing, and retrieval.
• Why it’s advanced: LlamaIndex excels via its NodeParsers and hierarchical indexing strategies (such as Small-to-Big / Parent-Child retrieval, where you embed fine-grained child chunks but feed broader parent nodes to the LLM). It also natively handles layout-aware parsing when paired with proper document loaders.
• Best for: Greenfield or heavy data-centric architectures that require robust metadata tracking and advanced node hierarchies.

3. LangChain (Best for Ecosystem & Structure-Aware Splitting)

• What it is: The ubiquitous modular framework for LLM application building.
• Why it’s advanced: It provides specialized splitters like MarkdownHeaderTextSplitter and RecursiveJsonSplitter that respect document syntax (headers, code blocks, JSON keys) rather than just raw character counts. It pairs cleanly with local or remote models if you want to implement Contextual Chunking (using an LLM pass to prepend global document/section context to individual chunks).
• Best for: Multi-step agentic setups or structured text (Markdown, HTML, codebases) where structural awareness is vital.

4. Unstructured / LlamaParse (Best for Complex Source Documents)

• What it is: Specialized parsing and ETL libraries/services designed to extract clean data from messy PDFs, tables, charts, and office documents.
• Why it’s advanced: True chunking failure usually happens before splitting—during bad PDF text extraction. Tools like Unstructured use layout detection and OCR models to preserve tables and section elements, ensuring your text splitter actually receives structured elements rather than scrambled text blocks.
• Best for: Enterprise document corpuses filled with dense PDFs, financial statements, or multi-column layouts.

Modern Advanced Strategies to Implement

Instead of asking which fixed size to use, consider these production-standard patterns:

• Semantic Chunking: Calculates the embedding distance between consecutive sentences; when the semantic distance exceeds a threshold (e.g., the 90th percentile of differences), it triggers a split.
• Parent-Child (Small-to-Big) Chunking: Indexes small chunks (~ 128 - 256 tokens) for precise vector matching, but dynamically expands the retrieved context passed to the generator to include the larger parent block (~ 1024 tokens).
• Contextual Enrichment / Retrieval: Uses a fast local or frontier LLM to prepend a short breadcrumb/summary of the overall document context directly into the metadata or header of each child chunk, slashing retrieval failure rates.

To help narrow down the ideal library and strategy for your setup, tell me:

• What document types are you primarily processing (e.g., messy PDFs, Markdown files, legal contracts, or raw codebases)?
• Are you looking for a lightweight standalone python package or a full ingestion framework like LlamaIndex/LangChain?