search and retrieval tuning tips for Kimi

[bytelab]

I hit almost the same issue last night.

[modrunner27]

I hit almost the same issue last night.

same here on my side.

If you post your exact settings, people here can probably narrow it down pretty fast.

[mintforge]

Did a bit of testing and thought I'd post the result here.

After a few runs on GB200, I ended up with decent performance, but the setup only really made sense once I adjusted around KV cache pressure.

Curious if anyone else landed in the same place.

[turbogrid11]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.

[macropilot]

That matches what I was seeing. The pattern around tool calling has been pretty similar for me.

[torchstation]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.

I hit almost the same issue last night.

[bytestack]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure.

[quietsupply11]

Good point. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[tinycontext]

I'd approach it in a boring order. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.