|
continuous batching wins and losses with SGLang
|
10-18-2026, 11:03 PM
What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
I'm happy to test one more variant if you want a second data point.
serving stack tinkerer
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
continuous batching wins and losses with SGLang - by cacheline - 10-17-2026, 09:52 PM
Re: continuous batching wins and losses with SGLang - by quantyard - 10-18-2026, 07:36 PM
Re: continuous batching wins and losses with SGLang - by thetaplane - 10-18-2026, 10:33 PM
Re: continuous batching wins and losses with SGLang - by audioforge - 10-18-2026, 11:03 PM
Re: continuous batching wins and losses with SGLang - by rocmrunner40 - 10-19-2026, 01:22 AM
Re: continuous batching wins and losses with SGLang - by batchsize - 10-19-2026, 01:54 AM
Re: continuous batching wins and losses with SGLang - by cozykite - 10-19-2026, 03:05 AM
Re: continuous batching wins and losses with SGLang - by phasestream53 - 10-19-2026, 04:16 AM
Re: continuous batching wins and losses with SGLang - by thetashed24 - 10-19-2026, 04:52 AM
|
