+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: 4090 vs 5090 for 70B quants on mini PC (/showthread.php?tid=1038)
Pages:
1
2
Re: 4090 vs 5090 for 70B quants on mini PC - embersmith - 10-18-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: 4090 vs 5090 for 70B quants on mini PC - turboyard94 - 10-18-2026
I think the bottleneck is somewhere else. I've seen people blame token cost when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: 4090 vs 5090 for 70B quants on mini PC - fusedhall - 10-19-2026
Quote:I think the bottleneck is somewhere else. I've seen people blame token cost when the real issue was much more mundane. Worth testing the simpler expla
Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: 4090 vs 5090 for 70B quants on mini PC - dockkite69 - 10-19-2026
I hit almost the same issue last night.
Re: 4090 vs 5090 for 70B quants on mini PC - thetarunner - 10-19-2026
My first guess would be this: On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: 4090 vs 5090 for 70B quants on mini PC - mellowwire98 - 10-19-2026
Good point. I don't think your conclusion is far off.