Inference › Inference Engines › vLLM / SGLang v
router in front of vLLM for Gemma

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
router in front of vLLM for Gemma
cacheline Offline
Senior Member
****
Posts: 341
Threads: 51
Joined: Sep 2026
Reputation: 0
#6
09-26-2026, 08:24 PM
I don't think that's the whole story. I get why that seems right, but on 5090 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
measure twice, benchmark three times
Find
Reply


Messages In This Thread
router in front of vLLM for Gemma - by batchsize - 09-20-2026, 04:46 PM
Re: router in front of vLLM for Gemma - by cacheline - 09-26-2026, 08:24 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)