+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: 4090 vs 5090 for 70B quants (/showthread.php?tid=256)
Pages:
1
2
4090 vs 5090 for 70B quants - thetarunner - 09-13-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of TensorRT-LLM and A6000 felt surprisingly usable once I tuned around prefill/decode split.
If someone wants the exact settings I can clean them up and post them.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Re: 4090 vs 5090 for 70B quants - torchshed - 09-13-2026
Thanks, that helps. I ended up in about the same place using LM Studio. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: 4090 vs 5090 for 70B quants - thetarunner - 09-14-2026
I'd approach it in a boring order. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: 4090 vs 5090 for 70B quants - sanevault - 09-14-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: 4090 vs 5090 for 70B quants - amberhall - 09-14-2026
Quote:I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the plain p
I'd approach it in a boring order. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. Turns out 'just one more test' was a lie.
Re: 4090 vs 5090 for 70B quants - cudapoint - 09-14-2026
Quote:[quote]I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: 4090 vs 5090 for 70B quants - paperport64 - 09-14-2026
Quote:[quote][quote]I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you kn
worth checking the driver first.
Re: 4090 vs 5090 for 70B quants - batchsize - 09-14-2026
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: 4090 vs 5090 for 70B quants - vrambandit - 09-15-2026
Thanks, that helps. The pattern around batch size has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: 4090 vs 5090 for 70B quants - thetarunner - 09-15-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.