vLLM vs SGLang for small teams on Threadripper box

[batchshed]

Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?

I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.