+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: 70B still worth the pain (/showthread.php?tid=989)
70B still worth the pain - cudapoint - 10-11-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on 5090, or squeeze more out of the stack and deal with the complexity.
What would you pick if you had to live with it for a few months?
Re: 70B still worth the pain - mintstream9 - 10-11-2026
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.
Re: 70B still worth the pain - dockbrook21 - 10-11-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: 70B still worth the pain - oxidestream - 10-11-2026
Quote:Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Thanks, that helps. That matches what I saw on 5090.
Re: 70B still worth the pain - macropoint - 10-11-2026
I hit almost the same issue last night.
Re: 70B still worth the pain - benchplease - 10-11-2026
Quote:I hit almost the same issue last night.
What exact setup are you running? Are you testing this on 4090 or something else?
Re: 70B still worth the pain - ggufwire22 - 10-11-2026
that fixed it for me too.
Re: 70B still worth the pain - cudapoint - 10-12-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: 70B still worth the pain - modrunner27 - 10-14-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. Once you know the plain path works, then add the faster path back in piece by piece.