+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Performance Tuning (https://www.datasculpting.ai/inference/forumdisplay.php?fid=42)
+--- Thread: consumer GPU serving economics for Gemma (/showthread.php?tid=878)
consumer GPU serving economics for Gemma - papermixer4 - 10-08-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: LM Studio on 5090, keeping prefix caching in mind from the start.
Feel free to tear it apart if you see something dumb.
Re: consumer GPU serving economics for Gemma - sparrowshed - 10-08-2026
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: consumer GPU serving economics for Gemma - rocmpath - 10-08-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: consumer GPU serving economics for Gemma - signalcraft14 - 10-10-2026
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: consumer GPU serving economics for Gemma - kernelmike - 10-10-2026
I tested this tonight. On 4090 with LM Studio, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: consumer GPU serving economics for Gemma - fastscope - 10-10-2026
subscribed, curious where this lands.
Re: consumer GPU serving economics for Gemma - edgeriver - 10-12-2026
I'd approach it in a boring order. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.