+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: 4090 vs 5090 for 70B quants (/showthread.php?tid=1309)
Pages:
1
2
4090 vs 5090 for 70B quants - batchwire - 10-16-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between PyTorch and MLX for Kimi. The interesting difference for me has been HBM constraints.
Trying to avoid rebuilding the whole thing twice.
Re: 4090 vs 5090 for 70B quants - cudapoint - 10-16-2026
Quote:Spent some time on this today and wanted to compare notes.
Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: 4090 vs 5090 for 70B quants - rocmriley - 10-17-2026
subscribed, curious where this lands.
Re: 4090 vs 5090 for 70B quants - signaltrack44 - 10-17-2026
That matches what I was seeing. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: 4090 vs 5090 for 70B quants - turbogrid11 - 10-17-2026
Quote:That matches what I was seeing. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
Re: 4090 vs 5090 for 70B quants - fastscope - 10-17-2026
A couple things I'd check first. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: 4090 vs 5090 for 70B quants - turbogrid11 - 10-17-2026
A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: 4090 vs 5090 for 70B quants - southspark - 10-18-2026
following this because I'm in the same boat.
Re: 4090 vs 5090 for 70B quants - rocmriley - 10-18-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: 4090 vs 5090 for 70B quants - batchshed - 10-19-2026
Good point. I ended up in about the same place using SGLang.