+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: vLLM / SGLang (https://www.datasculpting.ai/inference/forumdisplay.php?fid=36)
+--- Thread: router in front of vLLM: anyone testing this? (/showthread.php?tid=815)
router in front of vLLM: anyone testing this? - batchcraft - 10-06-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once continuous batching becomes important.
Interested in tradeoffs more than fan loyalty.
Re: router in front of vLLM: anyone testing this? - turboarch95 - 10-07-2026
That matches what I was seeing. I ended up in about the same place using TensorRT-LLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.