+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: 8B vs 14B sweet spot (/showthread.php?tid=498)
Pages:
1
2
Re: 8B vs 14B sweet spot - benchplease - 10-05-2026
Quote:worth checking the driver first.
Maybe, but I had the opposite result. I've seen people blame batch size when the real issue was much more mundane.
Re: 8B vs 14B sweet spot - quiettoken - 10-08-2026
What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: 8B vs 14B sweet spot - emberpeak - 10-11-2026
Maybe, but I had the opposite result. I get why that seems right, but on M3 Max the bottleneck felt elsewhere.