09-28-2026, 03:56 PM
Quote:Thanks, that helps. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more opti
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.
