+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: AWQ vs GPTQ vs GGUF right now (/showthread.php?tid=1014)
Pages:
1
2
Re: AWQ vs GPTQ vs GGUF right now - bytehall - 10-11-2026
I think the bottleneck is somewhere else. I get why that seems right, but on 5090 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: AWQ vs GPTQ vs GGUF right now - mintstation99 - 10-12-2026
Quote:I think the bottleneck is somewhere else. I get why that seems right, but on 5090 the bottleneck felt elsewhere. Could be a workload difference, but I
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: AWQ vs GPTQ vs GGUF right now - vrambandit - 10-12-2026
I hit almost the same issue last night.
Re: AWQ vs GPTQ vs GGUF right now - stackpilot - 10-12-2026
Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: AWQ vs GPTQ vs GGUF right now - cudaline - 10-13-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.