4090 vs 5090 for 70B quants on mini PC

[embersmith]

Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[turboyard94]

I think the bottleneck is somewhere else. I've seen people blame token cost when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.

[fusedhall]

I think the bottleneck is somewhere else. I've seen people blame token cost when the real issue was much more mundane. Worth testing the simpler expla

Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[dockkite69]

I hit almost the same issue last night.

[thetarunner]

My first guess would be this: On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

If you post your exact settings, people here can probably narrow it down pretty fast.

[mellowwire98]

Good point. I don't think your conclusion is far off.