+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: prefill/decode separation in practice (/showthread.php?tid=1398)
prefill/decode separation in practice - cacheline - 10-17-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once quant drift becomes important.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: prefill/decode separation in practice - localtinker - 10-19-2026
A couple questions before I guess too hard. Are you testing this on B200 or something else?
Re: prefill/decode separation in practice - oxidemixer - 10-19-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.