The best data ingestion pipeline for complex layouts and tables uses Docling or Unstructured combined with Markdown-based chunking and a Vision-Language Model (VLM).

Traditional text-extraction parsers (like PyPDF or pdfplumber) break when encountering multi-column text, embedded charts, or complex data tables. Modern pipelines preserve structural hierarchy before vectorization.

Recommended Pipeline Architecture

- Use **Docling** to parse complex PDFs directly into structured **Markdown** or JSON. It handles reading order, tables, and headers natively without losing spatial relationships.
- Alternatively, use **Unstructured** (`partition_pdf` ) to isolate text, tables, and images into separate distinct element arrays.[](https://google.com/goto?url=CAESVAHrOzAVU8ROB82gJZLy4T_WKBkGnTUPZbPfXmOa0MWYrPzsrp-ZS-DJKKvB6MsSe2oV1GvlJL31HplWQsayXNhUmFMUnkKgb9jnAtk9Eg50OPDDlQ) [[1]](https://google.com/goto?url=CAESVAHrOzAVU8ROB82gJZLy4T_WKBkGnTUPZbPfXmOa0MWYrPzsrp-ZS-DJKKvB6MsSe2oV1GvlJL31HplWQsayXNhUmFMUnkKgb9jnAtk9Eg50OPDDlQ)[[2]](https://google.com/goto?url=CAESgwEB6zswFSuz3NHjyjfJIEdStL6T4zZPSKAUSjePZsyqryBMH-QgugC2JXCHdcIFJBxVAz1sKgqkgPf8RWMRgf-tRNIPGA-za4dgu0gJ_HtuvTbcQGkYSmsTHVx-oS3Db6DSNThE6LrB2hntW4GEWnM5W7uMXZuFExwd1F7Avxurr4_bvQ)[[3]](https://google.com/goto?url=CAESUwHrOzAVmcRZe_j5DTvWz5K9WSUJMd9fV1E7JJ0KDRGPv-hJcSiT2_LLMcFJ8XaGpQhoccnf1SGKyL0RHvBwbBSQPV6fUiIfAGCz9llc-XfS3jIM)

- **Tables:** Convert tables into clean HTML or Markdown tables rather than raw text strips.
- **Images/Charts:** Pass figures and plots to an efficient Vision-Language Model (like **Gemini Flash** or **GPT-4o-mini** ) to generate descriptive text summaries.[](https://google.com/goto?url=CAESgwEB6zswFSuz3NHjyjfJIEdStL6T4zZPSKAUSjePZsyqryBMH-QgugC2JXCHdcIFJBxVAz1sKgqkgPf8RWMRgf-tRNIPGA-za4dgu0gJ_HtuvTbcQGkYSmsTHVx-oS3Db6DSNThE6LrB2hntW4GEWnM5W7uMXZuFExwd1F7Avxurr4_bvQ) [[1]](https://google.com/goto?url=CAESgwEB6zswFSuz3NHjyjfJIEdStL6T4zZPSKAUSjePZsyqryBMH-QgugC2JXCHdcIFJBxVAz1sKgqkgPf8RWMRgf-tRNIPGA-za4dgu0gJ_HtuvTbcQGkYSmsTHVx-oS3Db6DSNThE6LrB2hntW4GEWnM5W7uMXZuFExwd1F7Avxurr4_bvQ)[[2]](https://google.com/goto?url=CAESUwHrOzAVmcRZe_j5DTvWz5K9WSUJMd9fV1E7JJ0KDRGPv-hJcSiT2_LLMcFJ8XaGpQhoccnf1SGKyL0RHvBwbBSQPV6fUiIfAGCz9llc-XfS3jIM)[[3]](https://google.com/goto?url=CAESVAHrOzAVU8ROB82gJZLy4T_WKBkGnTUPZbPfXmOa0MWYrPzsrp-ZS-DJKKvB6MsSe2oV1GvlJL31HplWQsayXNhUmFMUnkKgb9jnAtk9Eg50OPDDlQ)

- Instead of arbitrary character splitting, use a **Markdown-aware chunk router** (such as LangChain's `MarkdownHeaderTextSplitter` ) to split documents cleanly by logical section headers, keeping parent tables intact.

- Convert the enriched text summaries, markdown blocks, and table representations into vector embeddings using a robust embedding model (e.g., OpenAI `text-embedding-3-small` or Hugging Face alternatives).
- Store vectors in a scalable vector database like **Milvus**, **Chroma** , or **Qdrant**.[](https://google.com/goto?url=CAESUwHrOzAVmcRZe_j5DTvWz5K9WSUJMd9fV1E7JJ0KDRGPv-hJcSiT2_LLMcFJ8XaGpQhoccnf1SGKyL0RHvBwbBSQPV6fUiIfAGCz9llc-XfS3jIM) [[1]](https://google.com/goto?url=CAESUwHrOzAVmcRZe_j5DTvWz5K9WSUJMd9fV1E7JJ0KDRGPv-hJcSiT2_LLMcFJ8XaGpQhoccnf1SGKyL0RHvBwbBSQPV6fUiIfAGCz9llc-XfS3jIM)[[2]](https://google.com/goto?url=CAESVAHrOzAVU8ROB82gJZLy4T_WKBkGnTUPZbPfXmOa0MWYrPzsrp-ZS-DJKKvB6MsSe2oV1GvlJL31HplWQsayXNhUmFMUnkKgb9jnAtk9Eg50OPDDlQ)

Top Tooling Options Compared

If you want, I can share:

• A Python code snippet showing how to set up Docling with LangChain
• A guide on parent-child retrieval to improve table search accuracy

Let me know how you would like to proceed.

For a large corpus of complex PDFs, I’d avoid the naïve pipeline of PDF → plain text → chunks → embeddings. You’ll lose layout semantics, table relationships, captions, and often reading order.

A better architecture is:

PDFs
  ↓
Document classification / page routing
  ↓
Layout-aware parsing + OCR
  ↓
Structured intermediate representation
  ├── headings / paragraphs
  ├── tables → HTML/Markdown + structured cells
  ├── figures → image + caption/description
  └── metadata: page, coordinates, section, document
  ↓
Semantic / structure-aware chunking
  ↓
Embeddings
  ↓
Vector DB + metadata store
  ↓
Hybrid retrieval + reranking

My recommendation

For this use case, Unstructured's layout-aware ingestion is a strong starting point, particularly if you want an open-source/self-hostable pipeline. Its current PDF partitioning supports auto, fast, hi_res, and OCR-based strategies; its auto routing can choose processing on a page-by-page basis rather than forcing the entire PDF through an expensive parser.

For complex PDFs, I'd configure the pipeline roughly like this:

• Simple text pages: fast extraction.
• Pages containing tables, figures, unusual layouts: high-resolution/layout-aware extraction.
• Scanned PDFs: OCR.
• Tables: preserve the table as a distinct object, ideally as HTML/structured cells rather than flattening it into prose.
• Figures: retain the image and associate it with its caption and surrounding text.
• Every element: retain document_id, page number, section/heading hierarchy, bounding box, and element type.

Unstructured's high-resolution PDF processing can produce table text plus an HTML representation that preserves rows/cells, which is much more useful downstream than treating a table as a paragraph.

Don't embed everything the same way

This is probably the most important design choice.

For example, instead of:

"Table 7. Revenue by region North America $4.2B Europe $3.8B..." store something like:

{
  "type": "table",
  "document": "annual_report_2025",
  "page": 87,
  "section": "Geographic Results",
  "html": "<table>...</table>",
  "summary": "Revenue by geographic region for FY2025.",
  "text_for_embedding": "FY2025 revenue by geographic region..."
}

Then maintain the actual table separately.

For retrieval, you can embed:

1. A table summary — good for semantic retrieval.
2. The table's normalized textual representation — useful for matching specific values/terms.
3. The actual table structure — retrieved alongside the chunk so the LLM can reason over rows and columns.

This avoids the classic RAG failure where the vector search finds the right table but the model receives a mangled sequence of cells.

Chunk around document structure

I would use hierarchical chunks, rather than fixed 500-token windows.

For example:

Document
└── Chapter 4: Financial Results
    └── 4.2 Revenue
        ├── paragraph
        ├── paragraph
        ├── Table 12
        └── paragraph

A chunk might therefore carry:

document_id
page_start / page_end
section_path
parent_heading
element_type
preceding_heading
following_heading

That metadata becomes extremely valuable for filtering and reranking.

Also consider keeping parent/child relationships:

parent chunk: "4.2 Revenue"
       ↓
child chunks:
  paragraph 1
  paragraph 2
  table 12
  paragraph 3

A search hit on a child can then retrieve the surrounding parent context.

For tables, use two retrieval paths

I'd actually build two complementary indexes:

Semantic index

paragraphs
headings
table summaries
figure descriptions

Structured/document index

tables
numbers
dates
identifiers
page locations
document metadata

Then combine:

vector retrieval
       +
BM25 / keyword retrieval
       +
metadata filtering
       ↓
reranker
       ↓
LLM context

This is substantially more robust for questions like:

"What was operating margin in Europe in 2024?" A pure vector search can be surprisingly bad at exact numeric/table queries.

What I'd use at scale

A practical production stack could be:

The key is not to make the vector database your canonical document store. Keep the original PDF and rich parsed representation elsewhere; vectors should be derived artifacts that you can regenerate.

One important caveat

Don't automatically send every page through the most expensive vision model. Current document-processing systems explicitly trade off speed, cost, and extraction quality; for example, Unstructured describes its fast strategy as dramatically cheaper/faster than image-to-text processing and its newer auto strategy as routing PDF pages according to their characteristics.

For a large corpus, I'd therefore use:

                    ┌─ simple text ──────→ fast parser
PDF → page router ──┼─ tables/layout ────→ layout parser
                    ├─ scanned ──────────→ OCR
                    └─ highly complex ───→ VLM
                                      ↓
                              canonical JSON
                                      ↓
                         structure-aware chunking
                                      ↓
                      ┌───────────────┴──────────────┐
                  vector index                 keyword index
                      └───────────────┬──────────────┘
                                  reranker
                                      ↓
                                     LLM

If your PDFs are especially table-heavy (financial reports, scientific papers, engineering documents, etc.), I'd make the table representation a first-class object rather than merely another chunk. That's usually the difference between a RAG system that looks good in demos and one that actually answers questions correctly from difficult PDFs.

For a large corpus of complex PDFs with multi-column layouts, figures, scanned pages, and tables, I’d use a layout-aware ingestion pipeline, not a simple PDF-to-text → chunk → embed workflow.

Recommended architecture

PDFs
  ↓
Object storage + document manifest
  ↓
PDF classification
  ├─ text-native/simple pages → fast text extraction
  ├─ complex layout → layout-aware parser
  └─ scanned/image pages → OCR / vision model
  ↓
Structured elements
  ├─ headings
  ├─ paragraphs
  ├─ lists
  ├─ tables → HTML/Markdown + structured representation
  ├─ figures → captions/descriptions
  └─ page/section metadata
  ↓
Semantic chunking
  ↓
Embeddings
  ↓
Vector DB + metadata store

My first choice: Unstructured

For this particular workload, docs.unstructured.io is a strong default because its PDF partitioning is explicitly layout-aware. Its hi_res strategy identifies document layout and can preserve tables as structured HTML; its newer routing can send simpler pages through faster extraction while using higher-quality processing for complicated pages.

For example, the critical part of an OSS pipeline looks roughly like:

from unstructured.partition.pdf import partition_pdf

elements = partition_pdf(
    filename="document.pdf",
    strategy="hi_res",
    infer_table_structure=True,
)

With table extraction enabled, table elements contain both text and an HTML representation that preserves rows/cells.

For a very large corpus, I'd actually avoid blindly running hi_res over every page. Use routing:

• Simple text-native page: fast extraction.
• Complex layout/table page: high-resolution/layout parser.
• Scanned page: OCR.
• Particularly difficult visual page/table: VLM-based extraction.

That matters because the fast path can be dramatically cheaper/faster, while the layout-aware paths preserve information that ordinary PDF text extraction destroys.

The important part: don't flatten tables

This is where many RAG pipelines go wrong.

Don't turn:

into an arbitrary text stream like:

Product Q1 10 Q2 20 Q3 30 A... Instead, preserve the table as a first-class object, ideally something like:

{
  "type": "table",
  "page": 42,
  "section": "Quarterly Results",
  "html": "<table>...</table>",
  "text": "...",
  "rows": [
    {"Product": "A", "Q1": 10, "Q2": 20, "Q3": 30}
  ]
}

Then create a retrieval representation of the table in addition to its canonical representation. For example, embed:

Quarterly Results — Product A: Q1 10, Q2 20, Q3 30. while retaining the original HTML/structured table for the LLM to inspect after retrieval.

Unstructured specifically supports producing HTML representations of extracted PDF tables.

Chunk at the semantic level

I would use section-aware chunks, rather than blindly splitting every N tokens.

A chunk should carry metadata such as:

{
  "document_id": "...",
  "page_start": 41,
  "page_end": 43,
  "section": "Quarterly Results",
  "element_types": ["paragraph", "table"],
  "parent_section": "Financial Performance",
  "chunk_id": "..."
}

Keep relationships between:

• document → section
• section → paragraphs
• section → tables
• table → page
• figure → caption
• chunk → neighboring chunks

This makes hierarchical retrieval and citations much better.

Embedding strategy

I'd generally create separate retrieval units for:

1. Normal prose
2. Tables
3. Figure/image descriptions
4. Headings/section summaries

Don't necessarily embed the raw HTML table. Generate a concise textual representation for embedding, while retaining HTML/JSON as the source representation.

For especially important tables, I'd consider two-stage retrieval:

query
 ↓
vector/BM25 retrieval
 ↓
candidate chunks + tables
 ↓
reranker
 ↓
LLM

Hybrid lexical + vector retrieval is particularly useful for PDFs because queries often contain exact identifiers, part numbers, regulation numbers, financial metrics, etc., where keyword retrieval can outperform embeddings.

What I'd deploy

If I were building this today, my baseline would be:

S3/GCS/Azure Blob → Unstructured → normalized document schema → semantic chunker → hybrid search/vector DB → reranker → LLM

With:

• Unstructured for PDF/layout/table extraction
• OCR/VLM fallback for pathological pages
• HTML/JSON as the canonical table representation
• page/section coordinates preserved as metadata
• semantic rather than fixed-size chunking
• hybrid BM25 + vector retrieval
• reranking before generation
• original PDF/page references retained for citations

The biggest design principle is: don't make vectors your canonical representation of the PDFs. Store the structured document separately and treat embeddings as an index over it. That lets you improve chunking, reranking, or embedding models later without re-ingesting the PDFs.

If you tell me roughly how many PDFs / total TB or GB, what kinds of documents they are (financial reports, engineering drawings, legal docs, scientific papers, etc.), and what vector DB you're using, I can recommend a concrete production architecture and component choices.

Hybrid lexical + vector retrieval is particularly useful for PDFs because queries often contain exact identifiers, part numbers, regulation numbers, financial metrics, etc., where keyword retrieval can outperform embeddings.

What I'd deploy

If I were building this today, my baseline would be:

S3/GCS/Azure Blob → Unstructured → normalized document schema → semantic chunker → hybrid search/vector DB → reranker → LLM

With:

• Unstructured for PDF/layout/table extraction
• OCR/VLM fallback for pathological pages
• HTML/JSON as the canonical table representation
• page/section coordinates preserved as metadata
• semantic rather than fixed-size chunking
• hybrid BM25 + vector retrieval
• reranking before generation
• original PDF/page references retained for citations

The biggest design principle is: don't make vectors your canonical representation of the PDFs. Store the structured document separately and treat embeddings as an index over it. That lets you improve chunking, reranking, or embedding models later without re-ingesting the PDFs.

If you tell me roughly how many PDFs / total TB or GB, what kinds of documents they are (financial reports, engineering drawings, legal docs, scientific papers, etc.), and what vector DB you're using, I can recommend a concrete production architecture and component choices.

When dealing with complex PDF layouts, multi-column reading orders, and intricate tables, traditional text-stripping tools (like PyPDF or basic PDFMiner) fail because they read across the page linearly rather than structurally.

The industry consensus for a robust data ingestion pipeline relies on layout-aware, vision-assisted, or multimodal parsing rather than raw text extraction.

Recommended Ingestion Pipeline Architecture

A production-grade pipeline for complex layouts and tables generally follows this 5-stage architecture:

1. Document Parsing & Layout Analysis: Extract structural elements (headers, footers, paragraphs, multi-column flows, tables, and figures) rather than raw characters.
2. Specialized Table & Image Processing: Convert tables into structured representations (HTML/Markdown) and use vision-language models (VLM) for figures/charts.
3. Semantic Chunking: Split content based on logical document hierarchies (section headers) rather than arbitrary character lengths.
4. Multi-Vector / Summary Enrichment: Generate textual summaries for tables and images alongside their raw structured data so the embedding model has high semantic context.
5. Embedding & Vector Storage: Embed the enriched chunks and store them in a vector database.

Top Open-Source & Managed Parsing Engines

Choosing the right parser for stage 1 and 2 dictates the success of the entire pipeline:

• Docling (IBM Research): Best overall for table-heavy and complex academic/corporate layouts. It performs deep layout analysis, accurately reconstructs complex tables into Markdown/JSON, handles multi-page tables well, and is MIT-licensed.
• Unstructured: Best for heterogeneous document sets. If your large set includes PDFs mixed with DOCX, PPTX, images, or HTML, the Unstructured Library handles 64+ formats and features a partition_pdf function that routes elements to text processors, OCR, or vision models.
• Marker: Best for high-speed, text-heavy extraction. If your complex PDFs are mostly dense text or research papers with standard multi-column layouts and speed is your primary constraint, Marker converts them to clean Markdown rapidly.
• LlamaParse (LlamaIndex): Best managed/cloud hybrid option. If you don't want to self-host heavy OCR/VLM models locally, LlamaParse is purpose-built by LlamaIndex to handle complex tables and layouts natively, returning clean markdown optimized for downstream chunking.

Suggested End-to-End Pipeline Implementation

A resilient Python pipeline utilizing a layout-aware parser and multi-vector retrieval typically looks like this:

1. Ingest & Parse via Docling or Unstructured: python```

Conceptual flow using Docling

from docling.document_converter import DocumentConverter

converter = DocumentConverter() result = converter.convert("complex_document.pdf") markdown_content = result.document.export_to_markdown()

Use code with caution.

2. Extract & Format Tables explicitly: Ensure tables are retained as HTML tags or clean markdown grids rather than flattened text strings.
3. Generate Summaries (Multimodal approach): Pass complex tables and figures through an affordable LLM/VLM to summarize their contents. This summary captures the intent of the table for vector search, while the raw HTML/Markdown table is passed to the LLM during generation.
4. Chunk & Embed: Use a semantic splitter via LangChain or LlamaIndex to group text under headings and embed using state-of-the-art embedding models.

<FollowUp> To help tailor this pipeline to your stack, tell me: * Are you planning to **self-host open-source models locally**, or do you prefer **cloud-managed APIs**? * What **vector database** or **orchestration framework** (LangChain, LlamaIndex, Haystack) are you leaning toward? </FollowUp> ```

For a large corpus of complex PDFs, I’d use a structure-first ingestion pipeline, rather than extracting plain text and chunking it. My default choice today would be Docling, with a fallback path for especially difficult pages. Docling explicitly preserves layout, reading order, table structure, OCR, and bounding-box metadata, and has a native structural chunker for RAG.

Recommended architecture

PDFs
  │
  ├── classify pages
  │     ├── born-digital/simple ──► fast parser
  │     ├── complex layout ───────► Docling layout parser
  │     └── scanned/image-heavy ──► OCR / VLM
  │
  ▼
Structured document representation
  │
  ├── headings / sections
  ├── paragraphs
  ├── tables → structured cells + Markdown/HTML
  ├── figures + captions
  ├── equations
  └── page/bounding-box metadata
  │
  ▼
Structure-aware chunking
  │
  ├── text chunks
  ├── table chunks
  └── figure/image descriptions
  │
  ▼
Embeddings
  │
  ▼
Vector DB + metadata store

Why this matters

Don't do:

PDF → extract_text() → 500-token chunks → embeddings

That tends to destroy table relationships, reading order, section hierarchy, and page context. Instead, preserve the document structure before you ever generate embeddings.

For tables in particular, I'd keep three representations:

1. Structured table — rows/cells/merged cells for downstream processing.
2. Markdown or HTML rendering — useful as embedding/LLM context.
3. Metadata — document ID, page, section, table ID, bounding box, caption.

Docling's TableFormer is specifically designed to recover table structure, including merged and borderless tables, while its RAG-oriented chunking preserves structural nodes and metadata.

My tool choices

Unstructured is also a good option, particularly if PDFs are only one part of a heterogeneous corpus. Its current pipeline can automatically route PDF pages between fast, high-resolution, and VLM processing, and its higher-quality modes preserve tables as structured/HTML representations.

The important optimization for a large corpus

Don't run the expensive parser on every page.

I'd implement page-level triage:

• Cheap text extraction first.
• Detect whether the page contains images, tables, unusual typography, multiple columns, or little/no extractable text.
• Send easy pages through the cheap path.
• Send difficult pages through Docling's full layout/table/OCR pipeline.
• Optionally send the worst-confidence pages to a VLM/cloud parser.
• Store the original PDF and structured intermediate representation so you can re-chunk/re-embed without reparsing.

This hybrid approach is increasingly preferable to treating every page identically.

One more important design decision

Don't make your vector database the source of truth.

Keep:

Object storage
  └── original.pdf

Document store
  └── DoclingDocument / JSON
       ├── sections
       ├── tables
       ├── figures
       └── coordinates

Vector DB
  └── embeddings + chunk IDs + metadata

That lets you change embedding models, chunk sizes, retrieval strategies, or LLMs without paying the PDF parsing cost again.

Bottom line: If I were building this from scratch, I'd start with Docling → structured JSON → structural/table-aware chunking → embeddings → vector DB, with page-level triage and a VLM/LlamaParse fallback for the pathological PDFs. Docling is particularly attractive when you need the whole pipeline self-hosted.

If you tell me roughly how many PDFs/pages you have, whether they're scanned, and what vector DB you're using (Pinecone/Qdrant/Weaviate/pgvector/etc.), I can recommend a concrete production architecture and chunking strategy.

When dealing with complex layouts, multi-column reading orders, and intricate tables, traditional text-extraction tools (like standard PyMuPDF, pypdf , or pdfplumber ) fail because they treat documents like uniform walls of text, scrambling rows, columns, and structural hierarchy.

The industry consensus has shifted toward vision-first and layout-aware multimodal parsers that output structured Markdown or HTML rather than flat text.

Top Pipeline Parser Choices

• LlamaParse : Best managed API for complex enterprise data (financial reports, nested tables, multi-column layouts). It uses Vision Language Models (VLMs) and agentic loops to reconstruct semantic structure into clean, chunk-friendly Markdown.
• Docling (IBM) : Best open-source, layout-aware framework. It natively handles tables, lists, and headings, outputting rich JSON or Markdown. It can run entirely locally for air-gapped or privacy-sensitive data, though it is heavier on compute.
• Reducto : Best specialized API for irregular or borderless tables, multi-page spanning tables, and complex document structures, outputting precise HTML/JSON alongside bounding box coordinates.
• Marker : Best fast open-source converter optimized for turning heavy tech documents and academic papers (with math formulas and complex formatting) into Markdown/JSON.
• Google Document AI / Azure Document Intelligence : Best enterprise hyperscaler solutions if you are already native to GCP or Azure and need high-throughput, compliant layout parsing trees.

Recommended Ingestion Architecture

A production-grade pipeline for complex PDF-to-vector conversion follows this sequence:

1. Document Router (Heuristic pre-screening) : Run a fast, lightweight check on the PDF. If it's a clean, single-column text PDF, use a fast native extractor to save costs. If it contains multi-columns, charts, or embedded data grids, route it to a VLM-powered layout parser (like LlamaParse or Docling).
2. Layout-Aware Parsing & OCR : Extract the page elements while retaining spatial awareness. Convert tables into HTML or structured Markdown (avoid flattening tables into raw space-delimited text, which destroys vector search relevance).
3. Hierarchical & Contextual Chunking : Rather than splitting text by arbitrary character counts, chunk based on the document tree (e.g., keeping a table paired with its immediate parent heading and accompanying caption/context).
4. Multimodal Embedding : Embed the structured text/table representations. For charts and graphics, pass visual crops to a VLM (like Qwen2.5-VL or GPT-4o) to generate descriptive text summaries that get embedded alongside the numeric data.

To help narrow down the ideal stack, tell me:

• Do you need a self-hosted/open-source solution or a managed cloud API?
• What is your approximate monthly page volume?

For a large corpus of complex PDFs, I’d avoid the “PDF → plain text → chunks → embeddings” approach. The biggest retrieval failures usually happen before embedding: broken reading order, flattened tables, lost headings, and OCR errors.

My default architecture

PDFs → layout-aware parser → structured intermediate representation → semantic chunks → embeddings → hybrid/vector index

For the parser, my first choice today would be Docling, particularly if you can run the ingestion infrastructure yourself. It explicitly handles reading order, multi-column layouts, tables/merged cells, OCR, bounding boxes, and structured document output. It also has a structural HybridChunker designed for RAG.

A good pipeline looks like this:

                ┌── ordinary text pages ───────┐
PDF ──► classify ── complex layout / tables ──► layout parser
                └── scanned pages ─────────────► OCR/VLM
                                                   │
                                                   ▼
                                    Structured document JSON
                                    ├── headings / hierarchy
                                    ├── paragraphs
                                    ├── tables + cells
                                    ├── figures + captions
                                    ├── page/bounding-box metadata
                                    └── reading order
                                                   │
                                                   ▼
                                      structure-aware chunking
                                                   │
                              ┌────────────────────┴─────────────┐
                              ▼                                  ▼
                       text embeddings                    table representation
                              │                                  │
                              └──────────────┬───────────────────┘
                                             ▼
                                  vector + keyword index
                                             │
                                             ▼
                                  hybrid retrieval / RAG

The important part: don't chunk tables like prose

For example, instead of turning this:

into a bag of tokens, retain the table structure and metadata:

{
  "type": "table",
  "document": "annual_report.pdf",
  "page": 47,
  "section": "Revenue",
  "headers": ["Product", "2024", "2025", "Change"],
  "rows": [
    ["A", "$10M", "$13M", "+30%"]
  ]
}

Then generate a searchable textual representation alongside the structured version. This gives your retriever something semantic to embed while preserving the original table for answer generation.

Why I like Docling here

Docling's current stack includes layout recognition, reading-order recovery, table structure extraction, OCR integrations, and a structured DoclingDocument; its table tooling is specifically intended to reconstruct things such as merged-cell and borderless tables.

It also means you aren't committing your canonical representation to Markdown. Keep the structured JSON as your source of truth, and generate Markdown/text only as a downstream representation for embedding or LLM context.

When I'd choose something else

Unstructured is particularly useful if PDFs are only one part of the corpus. Its PDF partitioner supports fast, hi_res, OCR, and table inference, and its newer auto-routing can send simple pages through fast extraction while directing complicated pages toward higher-quality processing.

One optimization I'd strongly recommend

Don't run the most expensive parser on every page.

For a large corpus, make ingestion page-adaptive:

1. Detect whether the page has embedded/extractable text.
2. Detect tables, images, multiple columns, unusual typography, etc.
3. Send simple text pages through a cheap extractor.
4. Send complex pages through Docling/VLM/layout analysis.
5. Send scans through OCR.
6. Store parser confidence and page metadata.
7. Reprocess only low-confidence pages.

That can dramatically reduce ingestion cost while preserving quality. Unstructured's current auto strategy follows essentially this philosophy, routing PDF pages differently depending on their content.

For the vector layer

I'd also use hybrid retrieval, not vector search alone:

BM25/keyword + dense embeddings + reranker

That's especially important for PDFs containing:

• part numbers
• legal clauses
• financial figures
• dates
• citations
• chemical/product names
• exact terminology

Dense embeddings are great at semantic similarity, but exact-match retrieval is often superior for those fields.

Bottom line: if you're building this yourself, I'd start with Docling → structured JSON → structure-aware chunking → hybrid retrieval, with a routing layer that sends only genuinely difficult pages through expensive OCR/VLM processing. That's a much more robust architecture than choosing an embedding model first and trying to compensate for bad PDF extraction afterward.

If you tell me roughly how many PDFs/pages you have, whether they're scanned, and whether you need to run everything locally, I can narrow this down to a concrete production stack (parser, chunking strategy, vector DB, and batch architecture).

When dealing with complex PDF layouts (multi-column text, floating figures, sidebars) and intricate tables (nested headers, merged cells, borderless grids), standard text-extraction libraries like pypdf or naive string-splitting pipelines will consistently fail. They flatten the text, scramble reading orders, and reduce tables to unreadable character salad.

The best data ingestion pipeline for this challenge relies on a Vision-First / Layout-Aware Parsing Architecture that converts documents into structured intermediate representations (like Markdown or HTML) before chunking and embedding.

Recommended Pipeline Architecture

[Raw PDFs] 
   ↓
[1. Classifier / Router] ──(Text-native vs. Scanned vs. Visual-heavy)
   ↓
[2. AI/Vision-First Parser] ──(LlamaParse / Docling / Reducto)
   ↓
[3. Structural Intermediate Format] ──(Markdown / HTML for tables)
   ↓
[4. Layout-Aware / Header-Aware Chunker] ──(Splits on semantic blocks, not tokens)
   ↓
[5. Embeddings & Vector DB]

Key Components & Tools

1. The Parsing Layer (Choose based on your deployment constraints)

• LlamaParse (Best Overall for LLM/RAG Readiness): Built explicitly for complex enterprise documents, financial filings, and nested tables. It treats parsing as a semantic reconstruction problem using vision intelligence and outputs clean Markdown/JSON.
• Docling (Best Open-Source / Self-Hosted): An exceptional layout-aware parser that handles multi-column flows, reads tables into HTML/Markdown, and extracts formulas into LaTeX. Ideal if you operate in privacy-first or air-gapped environments.
• Reducto (Best for High-Stakes Precision): Highly accurate on irregular, multi-page, or borderless tables, emitting robust HTML that preserves row and column spans (colspan / rowspan).
• Unstructured (Best for Format-Agnostic ETL): Excellent if your ingestion pipeline needs to scale across diverse file types (PDF, DOCX, PPTX) while segmenting elements into clean partitions.
• Azure AI Document Intelligence / Google Document AI (Best Enterprise Cloud-Native): Robust, highly compliant options with built-in deep learning models for table geometry and pre-trained forms.

2. Why HTML/Markdown Over Raw Text for Tables

Never flatten complex tables into comma-separated or space-delimited plain text.

• Instruct your parser to emit tables in HTML or structured Markdown.
• HTML preserves cell-to-header relationships via explicit tags, allowing the embedding model and the downstream LLM to accurately trace numeric values back to their specific column and row headers.

3. Layout-Aware Chunking

Standard fixed-size chunking (e.g., split every 500 tokens) slices straight through rows and headers. Your chunking strategy must be hierarchical and structure-aware:

• Split documents based on structural elements (Markdown headers #, ## , or HTML table boundaries) rather than arbitrary character counts.
• Retain parent metadata (such as the document title, section heading, and page number) and prepend it to each individual child chunk so context is never orphaned.

To help tailor this pipeline, could you share:

• What is your estimated document volume (e.g., hundreds vs. millions of pages)?
• Are you restricted to on-premises/air-gapped deployment , or is a cloud-managed API acceptable?

When converting complex PDFs—especially those featuring dense layouts, multi-column reading flows, and intricate nested or borderless tables—into vectors, traditional text-extraction libraries (like pypdf, pdfplumber , or basic PyMuPDF ) will consistently fail. They treat the page as a flat canvas, scrambling table cell coordinates and breaking multi-column reading order.

Modern ingestion pipelines use vision-first, layout-aware AI parsers that output structured Markdown or JSON before chunking and embedding.

Top Pipeline Recommendations

1. Best Managed Cloud API: LlamaParse (by LlamaIndex)

• Best for: Unpredictable layouts, complex nested/multi-page tables, and rapid deployment.
• How it works: It uses an agentic, multimodal approach to semantic reconstruction rather than basic text-box extraction. It handles split tables and mixed visuals remarkably well and natively outputs LLM-ready Markdown or structured JSON.
• Trade-off: Requires sending your documents to LlamaCloud, which may present a blocker if you have strict zero-data-retention or on-premises compliance needs.

2. Best Open-Source / Local Pipeline: Docling (by IBM Research) or Marker

• Best for: Data privacy, on-premise execution, and high-volume local processing without recurring API fees.
• How it works:
  • Docling excels at layout analysis, reading-order sequencing, and converting tables directly into structured data formats or Markdown. It also features native hierarchical chunkers.
  • Marker is purpose-built for fast, heavy-duty PDF-to-Markdown conversions, handling complex text blocks, equations, and code blocks cleanly.
• Trade-off: Local deployment requires adequate hardware (GPU acceleration recommended) and can occasionally hallucinate or misalign values on ultra-dense, borderless tables compared to cloud-hosted vision models.

3. Best Enterprise Compliance / Hyperscaler Choice: Google Document AI or Amazon Textract

• Best for: Enterprise environments already locked into a major cloud ecosystem with strict security certifications.
• How it works: Deeply trained machine learning models extract bounding boxes, key-value pairs, and table cells with high structural fidelity, mapping out geometric data per element.
• Trade-off: Setup is more rigid, and output usually requires custom formatting scripts to convert geometry payloads into clean, chunkable text for vector databases.

Recommended Ingestion Architecture

Regardless of the parser you select, avoid raw chunking on the output text. A production-ready pipeline should follow these sequential steps:

1. Classification & Routing: Classify incoming PDFs. Send clean, born-digital documents through a fast text extractor, and route visually rich/complex layout pages to your layout-aware parser (e.g., LlamaParse or Docling).
2. Structure-Preserving Parsing: Convert the PDF into structured Markdown . Markdown headers (#, ## ) and pipe tables (|--| ) preserve the hierarchical context necessary for downstream retrievers.
3. Semantic Table Handling: Do not blindly vectorize raw numerical table arrays, as LLMs can easily misalign or hallucinate numeric rows in standard embeddings. Instead, consider generating a natural-language semantic description/summary of each table to embed alongside the structural representation (storing the raw table JSON in a separate doc store).
4. Layout-Aware Chunking: Use a chunker that respects document hierarchy (such as Docling's hybrid chunker or LlamaIndex's hierarchical/node parsers) rather than a naive fixed-size character splitter, ensuring tables and their preceding headings stay intact.

To help narrow down the ideal choice, let me know:

• Do you have strict data privacy/on-premise requirements , or is a cloud API acceptable?
• Roughly how many pages are you planning to process?

For large collections of complex PDFs (financial reports, research papers, manuals, legal documents, invoices, etc.), the biggest determinant of retrieval quality is document parsing, not the embedding model. If your parser loses table structure or reading order, the vector database will faithfully store corrupted content. Studies comparing end-to-end RAG pipelines consistently find that hierarchy-aware parsing and chunking improve downstream QA accuracy more than swapping embedding models alone.

A production pipeline I'd recommend looks like this:

PDFs
   │
   ▼
Layout-aware parser
(Docling / LlamaParse / Document AI)
   │
   ▼
Structured document
(Markdown + JSON + layout metadata)
   │
   ▼
Semantic chunker
(section-aware, table-aware)
   │
   ▼
Embeddings
(text-embedding-3-large, bge-m3, etc.)
   │
   ▼
Vector DB
(Qdrant, Weaviate, Pinecone, Milvus, pgvector)

1. Use a layout-aware parser

Avoid simple text extraction libraries like PyPDF2 or plain PDF text extraction for complex documents—they often scramble multi-column layouts and tables.

Instead:

---

2. Preserve document structure

Don't flatten everything into one text blob.

Keep metadata such as:

• headings
• section hierarchy
• page numbers
• table boundaries
• captions
• reading order
• figure references
• document IDs

Good parsers can emit structured Markdown or JSON that retains this hierarchy, making downstream retrieval much more reliable.

---

3. Chunk semantically, not by tokens

Instead of:

every 500 tokens

prefer:

Section
 ├── subsection
 │    ├── paragraph
 │    ├── paragraph
 │    └── table

Good chunking strategies:

• one heading + its paragraphs
• tables as standalone chunks
• multi-page tables kept together
• overlap only between adjacent narrative sections

Hierarchical chunking generally outperforms naive fixed-size chunking.

---

4. Treat tables differently

A common mistake is embedding raw Markdown tables without context.

For each table, enrich it with information like:

Section:
Revenue by Region

Columns:
Region | FY24 | FY25 | Growth

Summary:
Annual revenue by geographic region.

This gives the embedding model meaningful semantic context beyond rows and columns. Research has shown contextualized table representations improve retrieval for table-centric questions.

---

5. Store rich metadata

For each chunk, include fields such as:

{
  "document": "...",
  "page": 17,
  "section": "Risk Factors",
  "chunk_type": "table",
  "heading": "Liquidity Ratios",
  "source": "annual_report.pdf"
}

This enables metadata filtering and better citations.

---

6. Recommended embedding models

Good current choices include:

• OpenAI text-embedding-3-large
• bge-m3
• e5-large-v2
• jina-embeddings-v4

If your corpus is multilingual, bge-m3 is particularly attractive because it supports dense, sparse, and multilingual retrieval.

---

My recommendations by use case

Open-source / self-hosted

• Docling
• Hierarchical chunking
• bge-m3
• Qdrant

This is an excellent stack if data privacy is important.

---

Highest retrieval quality

• LlamaParse
• Hierarchical chunking
• OpenAI embeddings
• Pinecone or Qdrant

If you're comfortable with a managed parsing service, this often produces the best results on complex layouts.

---

Enterprise-scale

• Google Document AI or Azure Document Intelligence
• Metadata-aware chunking
• Enterprise vector database (Pinecone, Milvus, Weaviate)

---

If you're processing tens or hundreds of thousands of PDFs, I'd also recommend adding:

• asynchronous ingestion with a work queue (e.g., Celery, Kafka, or RabbitMQ),
• parser result caching (avoid reprocessing unchanged documents),
• incremental re-indexing based on document hashes,
• OCR only when the PDF lacks extractable text, and
• validation checks (e.g., detect garbled tables or missing pages) before embedding.

These operational features often make a bigger difference to throughput and reliability than changing embedding models.