+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: 70B still worth the pain (/showthread.php?tid=1459)
Pages:
1
2
Re: 70B still worth the pain - fastquill74 - 10-19-2026
That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: 70B still worth the pain - macropoint - 10-19-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.
Re: 70B still worth the pain - kernelghost - 10-19-2026
I'd start with the simpler stack.
Re: 70B still worth the pain - plaingarden18 - 10-19-2026
subscribed, curious where this lands.
Re: 70B still worth the pain - phaseport36 - 10-19-2026
Interesting, I hadn't tried that. I ended up in about the same place using llama.cpp. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: 70B still worth the pain - routerunner - 10-19-2026
following this because I'm in the same boat.