+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: AWQ vs GPTQ vs GGUF right now on A6000 (/showthread.php?tid=757)
Pages:
1
2
Re: AWQ vs GPTQ vs GGUF right now on A6000 - cudahouse - 10-08-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: AWQ vs GPTQ vs GGUF right now on A6000 - benchplease - 10-10-2026
Appreciate the detail. I don't think your conclusion is far off.
Re: AWQ vs GPTQ vs GGUF right now on A6000 - marbletrack - 10-13-2026
My first guess would be this: If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: AWQ vs GPTQ vs GGUF right now on A6000 - cudaline - 10-14-2026
Quote:My first guess would be this: If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not gl
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal.
Re: AWQ vs GPTQ vs GGUF right now on A6000 - lablab - 10-14-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: AWQ vs GPTQ vs GGUF right now on A6000 - benchplease - 10-14-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there.