+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: prefill/decode separation in practice with LM Studio (/showthread.php?tid=793)
prefill/decode separation in practice with LM Studio - ggufvault - 10-06-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between TensorRT-LLM and SGLang for Mixtral. The interesting difference for me has been tail latency.
Trying to avoid rebuilding the whole thing twice.