Context budgeting: leave room for the answer, not just the prompt

Another document may appear to fit because the input alone is below the limit, but the response needs room too. A context budget makes allocations visible before you assemble an overcrowded request.

Capacity is not all document space

Check both the context window and output limit of the model you intend to use. The shared space can include input, output and reasoning tokens depending on the model; spare context does not remove a separate output limit.

Supply capacity, input, a response reserve and any overhead not already counted. These are planning values you enter, not an automatic measurement of your text or settings sent to a model.

References: OpenAI — Context windows and output limits

Do not count instructions and tools twice

If the input count already includes system instructions and tool definitions, do not add them again as overhead. Conversely, counting a paragraph alone does not necessarily cover those surrounding parts.

Write down what each figure includes: for example, input covers the document and question, while overhead covers instructions not yet counted. This is clearer than changing a margin until the balance turns positive.

Remaining capacity is arithmetic, not a quality certificate

If the total exceeds capacity, reduce material or the response reserve according to the task, then measure again. Do not remove an important instruction merely to make the request smaller.

A positive balance does not prove that every passage will be used accurately or that an answer will be correct and untruncated. Select material relevant to the question and evaluate the answer itself rather than filling space because it exists.

Example: an extra passage consumes the reserve

Suppose capacity is 12000 tokens: 7200 input, 3000 reserved for the response and 600 of uncounted overhead. Allocation is 10800, leaving 1200. This is a hypothetical capacity, not a specification for a particular model.

Adding a passage of 1800 tokens brings input to 9000 and total allocation to 12600: an overflow of 600. If the 600 instruction tokens were already included in input, enter zero additional overhead. Correct the accounting boundary rather than hiding the overflow.

References