+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: 70B still worth the pain with llama.cpp (/showthread.php?tid=1060)
70B still worth the pain with llama.cpp - batchscope72 - 10-12-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like token cost decides everything downstream.
Wondering if this is just where the whole space is heading now.
Re: 70B still worth the pain with llama.cpp - stackarch - 10-12-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: 70B still worth the pain with llama.cpp - phasescope - 10-12-2026
That matches what I was seeing. The pattern around prefill/decode split has been pretty similar for me.