prefill bottlenecks on longer prompts for Kimi

[tinycontext]

Thought I'd post a small recipe because a lot of threads ask similar things.

Maybe useful as a starting point: LM Studio on B200, keeping structured output in mind from the start.

Maybe useful as a starting point for newer folks.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

I expected the biggest difference to come from the model, but the stack around it mattered at least as much.

[macrotinker]

What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at HBM constraints. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.