+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: prefill/decode separation in practice (/showthread.php?tid=1126)
Pages:
1
2
Re: prefill/decode separation in practice - turboarch95 - 10-13-2026
Thanks, that helps. The pattern around reranking has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: prefill/decode separation in practice - cacheline - 10-16-2026
Quote:Thanks, that helps. The pattern around reranking has been pretty similar for me. I got better results once I stopped chasing one metric and just looke
following this because I'm in the same boat.
Re: prefill/decode separation in practice - oxidemixer - 10-18-2026
Quote:[quote]Thanks, that helps. The pattern around reranking has been pretty similar for me. I got better results once I stopped chasing one metric and jus
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at long context. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: prefill/decode separation in practice - turboarch95 - 10-18-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?