Inference › Inference Engines › vLLM / SGLang v
router in front of vLLM: anyone testing this?

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
router in front of vLLM: anyone testing this?
batchsize Offline
Member
***
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation: 0
#4
08-30-2026, 08:27 AM
worth checking the driver first.

If you post your exact settings, people here can probably narrow it down pretty fast.
throughput matters, but tail latency matters too
Find
Reply


Messages In This Thread
Re: router in front of vLLM: anyone testing this? - by batchsize - 08-30-2026, 08:27 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)