09-23-2026, 06:00 PM
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.
|
router in front of vLLM for Gemma
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
router in front of vLLM for Gemma - by batchsize - 09-20-2026, 04:46 PM
Re: router in front of vLLM for Gemma - by nanogarden - 09-20-2026, 05:37 PM
Re: router in front of vLLM for Gemma - by northgarden37 - 09-23-2026, 06:00 PM
Re: router in front of vLLM for Gemma - by batchcraft - 09-25-2026, 06:44 AM
Re: router in front of vLLM for Gemma - by cacheline - 09-26-2026, 08:24 PM
Re: router in front of vLLM for Gemma - by promptpoint - 09-26-2026, 07:08 PM
Re: router in front of vLLM for Gemma - by signalpeak - 09-26-2026, 09:07 PM
|
Users browsing this thread: 1 Guest(s)