Inference
desk setup thread for Gemma - Printable Version

+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: Off Topic (https://www.datasculpting.ai/inference/forumdisplay.php?fid=27)
+--- Thread: desk setup thread for Gemma (/showthread.php?tid=702)

Pages: 1 2

desk setup thread for Gemma - bytegarden73 - 10-03-2026

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on 4090, or squeeze more out of the stack and deal with the complexity.

Trying to avoid rebuilding the whole thing twice.

I care more about stability than squeezing the last possible tokens/sec out of it.

Re: desk setup thread for Gemma - kernelmike - 10-03-2026

Need a little more detail here. Are you testing this on 7900 XTX or something else?

Re: desk setup thread for Gemma - tinycontext - 10-05-2026

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

Re: desk setup thread for Gemma - benchplease - 10-05-2026

Quote:My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. A lot o

Thanks, that helps. I don't think your conclusion is far off.

Re: desk setup thread for Gemma - kernelarch - 10-05-2026

Interesting, I hadn't tried that. I ended up in about the same place using LM Studio.

Re: desk setup thread for Gemma - nanopatch - 10-05-2026

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals.

Re: desk setup thread for Gemma - vectorport - 10-05-2026

Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.

Re: desk setup thread for Gemma - batchshed - 10-07-2026

I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives.

Re: desk setup thread for Gemma - signaltrack44 - 10-08-2026

Quote:I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives.

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.

Re: desk setup thread for Gemma - labnest - 10-08-2026

I tested this tonight. On 7900 XTX with LM Studio, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.