containment for model-driven automation for Gemma

[kernelmike]

Thought I'd post a small recipe because a lot of threads ask similar things.

This isn't revolutionary, but the combination of Ollama and 7900 XTX felt surprisingly usable once I tuned around tool calling.

If someone wants the exact settings I can clean them up and post them.

[routeline23]

I don't think that's the whole story. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[vectorcrate98]

Tried to reproduce it just now. On dual 3090 with SGLang, I saw basically the same behavior.

[emberrunner]

What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[mangovault67]

Good point. That matches what I saw on 4090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[phasescope]

That matches what I was seeing. The pattern around tokens/sec has been pretty similar for me.