Long documents
How to split long text for an AI chat
The best chunk is not simply the largest piece an AI model accepts. A good split leaves room for the answer, keeps related material together, and gives the model enough continuity to connect one part to the next.
Start with the real input budget
Subtract the response you expect and any context already in the conversation from the advertised window. If the model has a 128K window, the chat already uses 20K, and you reserve 8K for the answer, only about 100K remains for new material.
Choose readable boundaries
Prefer headings, paragraphs, list boundaries, or complete sentences. Arbitrary character cuts can separate a claim from its evidence or break a code block halfway through a definition.
- Keep each part below the available budget, including its label.
- Repeat a small tail—often 100 to 300 tokens—when continuity matters.
- Label parts as Part 1 of N so order survives copy and paste.
- Tell the model whether to wait for every part before responding.
Avoid the context-stuffing trap
More context is not automatically better. Remove duplicate boilerplate, stale instructions, generated logs, and unrelated appendices first. A smaller, relevant packet is easier for a model to use and cheaper to send through an API.