Inference › Inference Engines › vLLM / SGLang v
router in front of vLLM for Phi

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
router in front of vLLM for Phi
dockridge Offline
Member
***
Posts: 73
Threads: 5
Joined: Aug 2026
Reputation: 0
#8
10-16-2026, 11:50 PM
A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Find
Reply


Messages In This Thread
router in front of vLLM for Phi - by cacheline - 10-16-2026, 03:04 AM
Re: router in front of vLLM for Phi - by dockridge - 10-16-2026, 11:50 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)