+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: AWQ vs GPTQ vs GGUF right now: worth it? (/showthread.php?tid=1430)
AWQ vs GPTQ vs GGUF right now: worth it? - cudaline - 10-17-2026
Did a bit of testing and thought I'd post the result here.
Short version: LM Studio behaved well for me, but only after I stopped ignoring workstation thermals.
Happy to post more details if anyone wants them.
Re: AWQ vs GPTQ vs GGUF right now: worth it? - vrambandit - 10-18-2026
That matches what I was seeing. That matches what I saw on GB200. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: AWQ vs GPTQ vs GGUF right now: worth it? - stackpilot - 10-18-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching. Not glamorous advice, but repeatable baselines save a lot of time.
Re: AWQ vs GPTQ vs GGUF right now: worth it? - cudaworks - 10-18-2026
My first guess would be this: On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: AWQ vs GPTQ vs GGUF right now: worth it? - benchplease - 10-19-2026
Quote:Appreciate the detail. The pattern around prefill/decode split has been pretty similar for me. I got better results once I stopped chasing one metric
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.
Re: AWQ vs GPTQ vs GGUF right now: worth it? - vrambandit - 10-19-2026
worth checking the driver first.
Re: AWQ vs GPTQ vs GGUF right now: worth it? - northquill - 10-19-2026
Appreciate the detail. The pattern around prefill/decode split has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.