+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: 70B still worth the pain (/showthread.php?tid=1461)
Pages:
1
2
Re: 70B still worth the pain - fastbench - 10-18-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: 70B still worth the pain - amberkite - 10-18-2026
I'd approach it in a boring order. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: 70B still worth the pain - routecrate - 10-18-2026
A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: 70B still worth the pain - fastscope - 10-18-2026
worth checking the driver first.
70B still worth the pain - turboscope - 10-19-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like NVFP4 decides everything downstream.
Maybe obvious to some of you, but it clicked for me this week.
Re: 70B still worth the pain - ggufwire22 - 10-19-2026
Thanks, that helps. I ended up in about the same place using LM Studio. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: 70B still worth the pain - mlxgarden - 10-19-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: 70B still worth the pain - batchsize - 10-19-2026
That matches what I was seeing. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: 70B still worth the pain - turboscope5 - 10-19-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: 70B still worth the pain - quietsupply11 - 10-19-2026
I'd start with the simpler stack.