+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: prefill/decode separation in practice for Kimi (/showthread.php?tid=1041)
Pages:
1
2
Re: prefill/decode separation in practice for Kimi - vramport - 10-17-2026
Quote:I hit almost the same issue last night.
What worked for me was pretty simple. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: prefill/decode separation in practice for Kimi - nanogarden - 10-18-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: prefill/decode separation in practice for Kimi - promptpoint - 10-19-2026
Need a little more detail here. Is this local-only or are you routing anything externally?