10-18-2026, 10:33 PM
That matches what I was seeing. I ended up in about the same place using TensorRT-LLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
|
continuous batching wins and losses with SGLang
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
continuous batching wins and losses with SGLang - by cacheline - 10-17-2026, 09:52 PM
Re: continuous batching wins and losses with SGLang - by quantyard - 10-18-2026, 07:36 PM
Re: continuous batching wins and losses with SGLang - by thetaplane - 10-18-2026, 10:33 PM
Re: continuous batching wins and losses with SGLang - by audioforge - 10-18-2026, 11:03 PM
Re: continuous batching wins and losses with SGLang - by rocmrunner40 - 10-19-2026, 01:22 AM
Re: continuous batching wins and losses with SGLang - by batchsize - 10-19-2026, 01:54 AM
Re: continuous batching wins and losses with SGLang - by cozykite - 10-19-2026, 03:05 AM
Re: continuous batching wins and losses with SGLang - by phasestream53 - 10-19-2026, 04:16 AM
Re: continuous batching wins and losses with SGLang - by thetashed24 - 10-19-2026, 04:52 AM
|
Users browsing this thread: 1 Guest(s)