Reading the RAG vendors, a companion to Building Enterprise RAG That Actually Works

For RAG on enterprise PDFs, structure beats fixed-size chunking

Reading the canonical chunking-strategy survey, and showing what each method actually does on a real PDF.

Source : Pinecone — Chunking Strategies for LLM Applications.

New here? This reading is a companion to the main series, Enterprise Document Intelligence: Building RAG That Actually Works. The series builds enterprise RAG as four bricks (parsing, question parsing, retrieval, generation) across five parts. This reading sits against the highlighted brick below.

The series architecture, all parts. This reading maps to the highlighted brick (the data model)

Pinecone’s Chunking Strategies for LLM Applications by Roie Schwaber-Cohen and Arjun Patel is the most-cited chunking guide on the web. It surveys five chunking methods. It names the precision-vs-richness trade-off. It covers chunk expansion as a post-processing step. The engineering content is real and the framing is widely used.

The disagreement is about ordering. The post leads with fixed-size chunking. That method became the field’s default, and it is the one that holds up worst on real documents. The defaults spread. Every RAG tutorial in the field now starts with RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200).

I am going to run four chunking strategies on a real PDF. With measurements, I will show why structure-first should be the default and why fixed-size should be the last resort.

1. The four strategies, applied to Attention Is All You Need

The source document is Attention Is All You Need by Vaswani et al. It is 15 pages, parsed with PyMuPDF. The full text comes out at around 39,500 characters. I apply four chunking strategies to the same text and report the actual numbers from the run.

the four strategies in code, applied to the same text

2. What the measurements actually look like

real measurements from one run of each strategy on the same PDF

Read the table. Four strategies, four very different behaviors:

• Fixed-size makes every chunk almost exactly 1000 characters. Predictable, uniform, blind to what the text says.
• Sentence-aware lands a bit smaller (median 936). It stops at the first sentence boundary that would push past 1000. It never exceeds 998 characters.
• Paragraph is uneven. Median 3111. Some paragraphs in the paper are very long. The related-work section runs into a single paragraph of 4255 characters. Some are short (777). Wide spread.
• Document structure is the most uneven of all. Some sections are just a heading (9 characters). Others are 7013-character section bodies. This unevenness is on purpose. Each chunk now matches a section of the paper.

3. What fixed-size actually cuts

The 1000-character boundary in the Attention paper sits inside a sentence. Here is the actual byte-1000 region:

The vertical bar is the chunk-0 / chunk-1 boundary. The sentence “Experiments on two machine translation tasks show these models to be superior in quality…” is cut between “show” and “these”. Chunk-0 ends with “show”. Chunk-1 starts with “these models to be superior…”.

This is the cost of counting bytes. Every fixed-size chunk in the field’s most popular tutorial has cuts like this scattered through it. The 200-character overlap softens the worst cases. The cut sentence reappears in chunk-1 with some context. But it does not prevent the cut. It just smears the damage.

actual cut counts. The 100% on fixed-size is not a typo

Fixed-size cuts every single chunk through a sentence boundary, because it never looks at sentences. Sentence-aware cuts zero. Paragraph cuts a few. Some paragraphs end with punctuation, some with a figure number. Structure-aware also cuts often. Section bodies in the Attention paper end on figure captions or table references, not on a period.

The mid-sentence cut count is a real measurement, not a thought experiment. It is the cost of the field’s default.

Where fixed-size fails on a structured document. A 1000-byte boundary does not know what it is cutting. On real documents it slices through the exact thing the reader asked about:

• A table of 12 rows is cut at row 7. Ask “sum the amounts” and retrieval pulls the slice with rows 1 to 7. The total is wrong and looks complete.
• A clause heading “Excess” lands at the end of one chunk and its value “500” at the start of the next. Ask “what is the excess?” and the chunk with the heading has no number, the chunk with the number has no heading.
• A definition splits across the boundary. Chunk-0 ends “for the purposes of this policy, a valid claim means”. Chunk-1 starts “any claim filed within 30 days”. Retrieval returns one half. The answer is half a definition.

Why it fails: fixed-size counts bytes, never structure. A table, a clause, a definition is a unit the author built. Cutting it at byte 1000 splits the unit at random. The fact is in the document. The chunking decided retrieval would see only part of it.

Here is the contrast run for real, both sides, on a public NIST PDF (the Cybersecurity Framework). On the left, a faithful reimplementation of the fixed-size chunking recipe. On the right, the series’ actual parser, docintel.parse_pdf.

both sides run for real on a public NIST PDF. Fixed-size makes arbitrary fragments with no address; docintel.parse_pdf keeps the document’s own lines, each citable.

4. What document structure gives you that fixed-size cannot

every chunk lands on a named section the author already wrote. The number prints on its own line in the PDF, so it gets its own column

Each chunk that document-structure splitting produces matches a named section of the paper. Chunk 0 starts with the Abstract. Chunk 1 with “1 Introduction”. Chunk 2 with “2 Background”. And so on. The structure was already in the document. The author put it there. Fixed-size chunking throws it away.

The series goes one level finer than section-level chunks. The parser hands you a line_df: one row per text line, with the page, the line number on the page, the bounding box, and the line text. No chunker invented any of it. The author put the line breaks in. This is the second of the two layers a PDF carries: the geometry the author committed, not the byte stream a splitter slices (Article 5A (what to read in a PDF)).

the parser hands the rows directly, page and line are plain integers and every line keeps its bounding box. The chunker is downstream of this, not upstream

Say a user asks “what does the paper say about positional encoding ?”. Document-structure retrieval finds the “3.5 Positional Encoding” chunk by metadata. Fixed-size retrieval has to hope the embedding of the user query lands close to whichever 1000-byte slice holds the answer. The first approach is one filter and one lookup. The second is a similarity search in 1536 dimensions.

The same gap shows up on enterprise documents, with higher stakes. Ask a 40-page insurance policy “what is the excess on a water-damage claim?”. With structure-aware chunks, retrieval jumps to the “Excess” clause or the perils schedule by its heading. With fixed-size chunks, the excess figure might sit at byte 23,140, halfway through a slice that starts mid-sentence in an unrelated clause. The number is in the corpus. The chunking decided whether retrieval can find it cleanly. Mapping the user’s wording to the policy’s wording is the question parser’s job (Article 6 (question parsing)). Putting the answer in a clean, addressable chunk is the data model’s job (Article 5B (data model)).

5. What to keep, and what to build

The Pinecone chunking guide is the field’s reference. The chunking sweet-spot trade-off is real. The survey of methods is broad. The diagrams are clear. The series adopts the chunk expansion vocabulary (Article 7, retrieval, coming soon) and treats the post as the canonical chunking reference.

The one thing worth flipping is the ordering. Fixed-size chunking should not be the first method shown. It came last on every measurement I just ran on a real PDF. It cuts every chunk mid-sentence. It ignores document structure. It produces uniform-but-meaningless slices that the embedding has to recover from.

Document-structure chunking is the right default for any document with a TOC, headings, paragraphs, table rows, or list items. Sentence-aware is the right fallback for documents without structure. Fixed-size is the last resort, used when nothing else is available.

A reader who finishes the Pinecone post and reaches for the first method shown will build a chunker on a broken assumption. The series’ structure-first parsing (Article 5B (data model)) keeps page-level as the cheap default and drills to line-level only when retrieval needs it. Use the boundaries the author put in, not the byte offsets a chunker invented.

The post is good. Putting fixed-size as the lead method is wrong. The measurements above are why.

For the alternative the series ships, see Article 5B (data model) for the relational line_df and page_df tables, and Article 7 (retrieval), coming soon, for the page-then-line drill-down.