+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: 70B still worth the pain with llama.cpp (/showthread.php?tid=747)
Pages:
1
2
Re: 70B still worth the pain with llama.cpp - amberpatch25 - 10-11-2026
Good point. That matches what I saw on mini PC. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: 70B still worth the pain with llama.cpp - llm_wren - 10-12-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: 70B still worth the pain with llama.cpp - mellowsupply - 10-12-2026
same here on my side.
Re: 70B still worth the pain with llama.cpp - mintstation99 - 10-13-2026
A couple things I'd check first. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: 70B still worth the pain with llama.cpp - orbitbench6 - 10-13-2026
Quote:A couple things I'd check first. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of
What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: 70B still worth the pain with llama.cpp - cacheline - 10-16-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?