+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: router in front of vLLM with vLLM (/showthread.php?tid=313)
router in front of vLLM with vLLM - rocmrunner40 - 09-16-2026
Did a bit of testing and thought I'd post the result here.
Short version: vLLM behaved well for me, but only after I stopped ignoring HBM constraints.
Nothing scientific here, just practical notes from a real setup.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
The part that surprised me was how much the boring details mattered more than the model choice.
Re: router in front of vLLM with vLLM - mangofield33 - 09-16-2026
Can you fill in a bit more context? Are you testing this on B200 or something else?