R6.4 · LESSON 4 OF 6

Semantic boundaries

Create source-traceable chunks, calculate overlap and explain ingestion choices that preserve answer-bearing context.

THE IDEA

Semantic boundaries

Instead of cutting at a fixed count, cut at natural boundaries: headings, paragraphs, list items. Pack whole paragraphs into chunks up to a limit, and split a paragraph only when it alone exceeds the limit. Paragraphs of 80, 120, 60 and 150 words at a 200-word limit become chunks of 200, 60 and 150. These rules are heuristics: some questions need evidence from two sections, which no single chunk holds.

maximum words per chunk200

P1: 80P2: 120P3: 60P4: 150

greedy packing at 200 words: chunks of 200, 60, 150 words (3 chunks) paragraphs are kept whole when they fit; one longer than 200 is split into pieces of at most 200 topic boundaries are a heuristic: an answer that needs two sections is still split across chunks

A 350-word paragraph at a 200-word limit becomes pieces of 200 and 150 words. Evaluate boundary rules on questions with known evidence, not on how tidy the chunks look.

Change the limit and watch the packing.

Predict first. Can a question require evidence across two well-formed sections?

Chapter context

Retrieval can only find information represented in its candidate chunks. Chunking chooses the unit of retrieval; it should preserve enough context to answer a question while remaining small enough to rank and use effectively.

TRY IT YOURSELF

Part 1. Do it hands-on

Pack paragraphs by hand, then write the packer and an evidence check.

maximum words per chunk200

P1: 80P2: 120P3: 60P4: 150

A 350-word paragraph at a 200-word limit becomes pieces of 200 and 150 words. Evaluate boundary rules on questions with known evidence, not on how tidy the chunks look.

chunks at 200 words chunk sizes pieces for a 350-word paragraph answer may span two sections? (yes/no)

To do: Paragraphs of 80, 120, 60 and 150 words are packed greedily into chunks of at most 200 words (never splitting a paragraph that fits). How many chunks?
To do: The chunk sizes in words, comma separated?
To do: A single paragraph of 350 words does not fit in 200. Splitting it into pieces of at most 200 gives how many pieces?
To do: Can a question require evidence from two well-formed sections (so no single chunk holds the whole answer)? Type yes or no

Part 2. Do it in code

pack_paragraphs(paragraphs, max_words): paragraphs is a list of strings. Pack them greedily, in order, into chunks of at most max_words words: add a paragraph to the current chunk if it still fits, otherwise start a new chunk. A paragraph longer than max_words is first flushed from the current chunk, then split into consecutive pieces of at most max_words words, each piece its own chunk. Return the chunks as strings (paragraphs in one chunk joined by "\n\n").
chunk_sizes(chunks): the word count of each chunk.
evidence_in_one_chunk(chunks, evidence): the index of the first chunk containing the evidence string, or None.
answerable(chunks, evidence_parts): True if some single chunk contains ALL the evidence strings.

def pack_paragraphs(paragraphs, max_words):
    pass

def chunk_sizes(chunks):
    pass

def evidence_in_one_chunk(chunks, evidence):
    pass

def answerable(chunks, evidence_parts):
    pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Can a question require evidence across two well-formed sections?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

For document length N, maximum chunk length C, overlap O and stride $S=C-O>0$ , a coverage-stopping scheme uses

K=1+\left\lceil\frac{\max(0,N-C)}{S}\right\rceil\quad(N>0).

Stop once a chunk reaches the document end; blindly iterating start positions can create redundant trailing chunks. Define empty-document behavior separately.

Worked example

For N=1000,C=200,O=50, stride is 150 and K=1+ceil(800/150)=7. Starts are 0,150,300,450,600,750,900; the final chunk contains 100 tokens. Total stored token positions are 6×200+100=1300 because overlap duplicates context.

Reference code

The following example uses the Python standard library and does not call a remote service.

from math import ceil
N,C,O=1000,200,50
chunks=[]
start=0
while start < N:
    end=min(start+C,N)
    chunks.append((start,end))
    if end == N: break
    start += C-O
assert len(chunks) == 1+ceil(max(0,N-C)/(C-O)) == 7
assert sum(end-start for start,end in chunks) == 1300
assert chunks[-1] == (900,1000)

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

Overlap is not new evidence. Chunk-size advice depends on corpus and retrieval task. Retrieved passages need provenance and versioning even when their text appears self-contained.