CPU-only fallback performance for Gemma

[macrotinker]

Good point. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[vramscope]

What exact setup are you running? Is this local-only or are you routing anything externally?

[quietsupply11]

What exact setup are you running? Is this local-only or are you routing anything externally?

I'd approach it in a boring order. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.

[circuitkite]

Good point. That matches what I saw on dual 3090.

[vramridge]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.