|
continuous batching wins and losses with SGLang
|
10-19-2026, 03:05 AM
My first guess would be this: If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.
team red, team patient
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
continuous batching wins and losses with SGLang - by cacheline - 10-17-2026, 09:52 PM
Re: continuous batching wins and losses with SGLang - by quantyard - 10-18-2026, 07:36 PM
Re: continuous batching wins and losses with SGLang - by thetaplane - 10-18-2026, 10:33 PM
Re: continuous batching wins and losses with SGLang - by audioforge - 10-18-2026, 11:03 PM
Re: continuous batching wins and losses with SGLang - by rocmrunner40 - 10-19-2026, 01:22 AM
Re: continuous batching wins and losses with SGLang - by batchsize - 10-19-2026, 01:54 AM
Re: continuous batching wins and losses with SGLang - by cozykite - 10-19-2026, 03:05 AM
Re: continuous batching wins and losses with SGLang - by phasestream53 - 10-19-2026, 04:16 AM
Re: continuous batching wins and losses with SGLang - by thetashed24 - 10-19-2026, 04:52 AM
|
