|
router in front of vLLM for Gemma
|
09-26-2026, 09:07 PM
Quote:I don't think that's the whole story. I get why that seems right, but on 5090 the bottleneck felt elsewhere. Not saying you're wrong, just that I woul A couple things I'd check first. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
mostly local
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
router in front of vLLM for Gemma - by batchsize - 09-20-2026, 04:46 PM
Re: router in front of vLLM for Gemma - by nanogarden - 09-20-2026, 05:37 PM
Re: router in front of vLLM for Gemma - by northgarden37 - 09-23-2026, 06:00 PM
Re: router in front of vLLM for Gemma - by batchcraft - 09-25-2026, 06:44 AM
Re: router in front of vLLM for Gemma - by cacheline - 09-26-2026, 08:24 PM
Re: router in front of vLLM for Gemma - by promptpoint - 09-26-2026, 07:08 PM
Re: router in front of vLLM for Gemma - by signalpeak - 09-26-2026, 09:07 PM
|
