When counting characters is the right answer

Someone reads one article about structural chunking, writes a markdown parser, a table detector and a code-aware splitter, re-indexes, and retrieval barely moves. Then they look at the corpus and discover that most of it is plain-text exports, pasted email bodies and PDFs that came out as one continuous paragraph.

Fixed-size splitting is the thing everyone recommends against, including me. It is also correct more often than its reputation suggests, and the sensible position is to know exactly when.

What counting actually guarantees

Three properties, and each one is worth something.

It works on any input. No parser, no format detection, no per-type handling, no failure mode where the parser throws on a malformed document and the ingest job dies. Feed it bytes, get chunks. On a corpus you do not control and cannot characterise, this is not a small thing.

Chunk sizes are uniform. Every chunk is roughly the same length, so every vector is compressed by roughly the same amount. Structural splitting produces a two-line chunk next to a nine-page one, and those two are not comparable on the same similarity scale. Uniformity is a genuine advantage that almost nobody counts on the fixed-size side of the ledger.

It is trivially reproducible. Given the same input and the same parameters you get the same chunks, forever, with no dependence on a parser version or a heuristic that fires differently on Tuesday. When you need to explain why a chunk exists, “characters 4,000 to 5,000 of document 12” is a complete explanation.

Where it is the right call

When you do not yet know what your corpus is. Fixed-size is the baseline. You need one, because “structural splitting helped” is a claim about a comparison, and without a baseline you are guessing. Build the dumbest possible splitter first, index with it, and keep it around as the thing every later strategy has to beat.

When the structure is absent or lying. Plain-text exports. Documents where every heading is a <b> tag applied to whatever the author felt like emphasising. Word files where hierarchy was done with manual indentation. Parsing these faithfully reproduces nonsense — a level-3 heading under a level-1 with nothing between, or a “section” that is one bold sentence in the middle of a paragraph. Counting characters at least fails predictably.

When the corpus is heterogeneous and enormous. A dispatcher with format-specific handling for every type in a million-document corpus is a real engineering commitment, and it has to be maintained against whatever new formats arrive. Doing that work only for the document classes that matter, and counting characters on the rest, is a reasonable allocation. Deciding which classes matter is what routing is for.

When the document is already short. A document that fits comfortably inside one chunk does not need a strategy. The splitter’s only job is to not split it, which every splitter does correctly.

Bad counting versus careful counting

Most of what people hate about fixed-size splitting is not the counting. It is three or four implementation shortcuts that ship with it by default. Here is the same source, cut naively and then cut carefully at the same nominal size.

=== NAIVE: hard cut at the character count ===
--- chunk 1 (chars 0–260) ---
## Webhook delivery

Events are delivered as POST requests to the URL configured
in your dashboard. Delivery is attempted up to five times.
A delivery is considered successful when your endpoint
returns any 2xx status within the timeout window. Non-2xx
responses and time
--- chunk 2 (chars 260–520) ---
outs are both treated as failures and
retried on the schedule below. After the fifth failure the
event is dropped and a delivery-failure notification is
sent to the account owner.

=== CAREFUL: same size, four adjustments ===
--- chunk 1 ---
[API Reference → Webhook delivery]

Events are delivered as POST requests to the URL configured
in your dashboard. Delivery is attempted up to five times.
A delivery is considered successful when your endpoint
returns any 2xx status within the timeout window.
--- chunk 2 ---
[API Reference → Webhook delivery]

Non-2xx responses and timeouts are both treated as failures
and retried on the schedule below. After the fifth failure
the event is dropped and a delivery-failure notification is
sent to the account owner.

Same splitter, same nominal length, same absence of a parser. The difference is that chunk 2 in the second version is a complete statement that knows what document and section it belongs to, and chunk 2 in the first version opens with the string outs are both treated as failures.

Note what did not change: nothing here required understanding markdown. The heading prefix came from a regex for lines starting with #, tracked as a running variable. That is an afternoon of work, not a parser.

The four adjustments, as a procedure

1. Align boundaries to sentence ends. Find the target offset, then move the cut to the nearest sentence boundary within some tolerance. This eliminates the mid-word and mid-sentence fragments that cause most of the visible damage, and it costs nothing at index time.
2. Track and prepend the nearest preceding heading. Even a crude regex for markdown hashes, or for short all-caps lines, or for lines followed by a row of dashes. This is the single highest-value change and it is why the argument for structural splitting partly applies even when you are still counting.
3. Never cut across a document boundary. Split each document independently. Concatenating a corpus and then splitting produces chunks that straddle two unrelated documents, and they are worse than anything else on this page.
4. Treat a few things as atomic even without a parser. A fenced code block, a run of consecutive lines beginning with |, a block of lines beginning with a bullet. Refusing to cut inside those three patterns is a handful of conditions and it removes the failures that annoy readers most.

Add sentence-aligned overlap only after these, not instead of them — overlap is insurance against a badly placed boundary, and steps 1 and 2 place boundaries better, so you need less of it.

What it still breaks

Deep hierarchy. A document with meaningful four-level nesting has relationships between sections that a running-heading regex cannot represent, and each chunk will carry at best the nearest heading rather than the path.

Anything where the unit of meaning is not a span of prose: tables with stacked headers, spreadsheets, source files, transcripts where the atomic unit is a speaker turn. Careful counting makes prose acceptable. It does nothing for these.

Uneven relevance within a chunk. A carefully cut 1,000-character chunk still contains whatever happened to be adjacent, which may be the end of one topic and the start of another. Counting cannot know where a topic ends, which is the entire point of every other strategy on this site.

How to tell whether you need more

Sample the chunks and read them, which is the review that finds most of these. Then ask a narrower question than “are these good”: for each chunk, does it end where a human would have ended it?

If most boundaries land at plausible places, counting is working on this corpus and your effort belongs elsewhere — metadata, or the one document class that is clearly mangled. If most boundaries land mid-thought, the corpus has structure you are not using, and now you have a reason to write the parser rather than a blog post’s opinion.