Does `thinking.keep` decide whether historical `reasoning_content` counts toward the prefix cache on kimi-k2.6 and later?

[km_alf]

On authorship: this question was written by the AI assistant that does most of the
development work on my addon, on my behalf and posted under my name. Flagging it
because you should know when you’re reading machine-written text.

Hi,

I maintain FreeCAD AI, an open-source CAD
assistant addon. It talks to Kimi over the OpenAI-compatible /v1/chat/completions
endpoint, kimi-k2.6, multi-turn with tool calls in most turns.

I just finished fixing a prefix-cache bug on our side: we were re-rendering history
without the reasoning_content we had actually sent, so the prefix diverged at the
first assistant message of every turn. That is the same failure yuikns described in
“Cached tokens drop when tool has interleaved thinking”
— thanks for that thread, it saved me a lot of guessing. We now store and replay
reasoning_content verbatim.

What I can’t work out from the docs is how that squares with thinking.keep.

1. On kimi-k2.6 with thinking.keep unset (the null default), is historical
reasoning_content dropped before longest-prefix matching, or after?

The field reference says the default means “The server ignores reasoning_content
from historical turns”. If “ignores” means the server strips it while re-assembling
the context, then a client that faithfully replays every reasoning block still gets a
prefix divergence at each turn boundary — because the bytes the server matches against
are not the bytes the client sent — and keep: "all" would be the only way to keep the
cache running past turn N. If “ignores” only means the model doesn’t attend to it, then
replaying it is enough and keep is irrelevant to caching. Which is it?

(For completeness: we currently send no thinking object at all, so we’re on the
default.)

2. If the answer is “set keep: \"all\"”, what happens to histories that predate
the change? A saved conversation can contain an assistant message with tool_calls
and no reasoning_content, because the client never stored it. Is that a hard error
under keep: "all", and if so, is there a supported way to migrate — drop the whole
turn, drop the tool_calls, send an empty string?

3. prompt_cache_key: the docs recommend a per-session id for multi-turn agents.
Should it stay fixed across a context compaction — where the client summarises old
turns and the prefix genuinely changes — or should compaction start a new key?

4. Block granularity. Measuring one of my own sessions (28 request pairs), the
reported cached_tokens fit floor(previous_request_prompt_tokens / 2048) * 2048 in
24 of them, and every miss was a turn boundary of the kind above. Only two distinct
values ever appeared, 12288 and 14336. Is ~2048 tokens the block granularity, i.e. is
there a size below which stabilising a prefix simply cannot show up in the bill? I ask
because I’d like to tell my users honestly when this optimisation is worth anything —
my own test sessions peaked around 15k prompt tokens and showed no measurable saving.

Happy to post before/after numbers from a longer session if that’s useful to anyone.

Thanks,
Alfred

[yuikns]

Hi Alfred,

Thanks for the detailed write-up (and the honest disclosure). Taking your questions in order:

1. Does thinking.keep decide whether historical reasoning_content counts toward the prefix cache?

On kimi-k2.6, thinking.keep accepts "none" or "all", and an unset (null) value defaults to "all" — so you’re already on "all" even without sending the object.

The disposition of reasoning_content happens during request preprocessing / tokenization, i.e. before prefix-cache matching, which runs on the token blocks that stage produces. One k2.x-specific detail explains your observations: the k2.x series runs in interleaved thinking mode by default, and in that mode some thinking from earlier turns may not enter the model at all — and content that never enters the model is never tokenized or counted, so it cannot contribute to the prefix cache either. That’s the mechanism behind the turn-boundary cache drops you measured; it’s structural to this model generation, not a client-side bug.

That said, the actionable recommendation is not to start carving your history around this behavior. Do the opposite: persist every reasoning_content and replay it verbatim, and control the behavior through the request parameter (thinking.keep) rather than by editing what you send. What actually enters the model is decided server-side, per model and per request — clients shouldn’t replicate that logic, and a faithfully preserved history is also what makes migrating to future models painless. (For contrast: k3 is no longer interleaved thinking, only supports keep: "all", and never prunes any previous reasoning_content; its cache-block design also differs from k2.x for architectural reasons.)

2. Histories that predate the change

Missing reasoning_content on a historical turn is not a hard error — consistent with your observation that your pre-fix requests succeeded. (There is a sanity check on our side that at least one turn carries reasoning_content; it exists as fool-proofing, not as per-turn validation.)

The supported migration is to backfill a reasoning_content on old assistant turns — an empty placeholder is acceptable. The one thing you should not do is drop the tool_calls: that directly breaks the model’s reasoning chain, since the surrounding thinking leads into and builds on those calls.

And to widen the point beyond this migration question: none of this is specific to tool-calling turns. In ordinary multi-turn interactions you should likewise preserve the reasoning_content of every turn wherever possible — our benchmarks show a clear, measurable drop in response quality for turns where it is absent. Backfilling an old turn restores validity, not quality; going forward, persist everything.

3. prompt_cache_key

It’s a routing hint, not a cache namespace. The API layer uses it — alongside load-balancing and other factors — to pick which underlying cluster serves the request. One key per user session is the right granularity. No need to rotate it for tool calls, and no need to rotate it after a compaction either: the compacted prefix would be a first-time miss regardless of key, and a stable key keeps your routing stable.

4. Block granularity

There is no single global value here: granularity is configured per cluster and per model architecture — some of our configurations use 256-token blocks — and text vs. multimodal inputs also behave differently (images and similar content may be accounted for as a whole unit). So a single-session fit like your 24/28 match to a 2048 floor shouldn’t be read as a global constant; stable prefix sizes, plus whichever cluster your requests happened to land on, explain it just as well. At ~15k prompt tokens, granularity isn’t the obstacle anyway — the structural behavior in (1), plus routing variance, are the likelier explanations for seeing no measurable saving.

Happy to look at before/after numbers from a longer session if you post them.