+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: AWQ vs GPTQ vs GGUF right now (/showthread.php?tid=535)
AWQ vs GPTQ vs GGUF right now - northquill - 09-27-2026
Did a bit of testing and thought I'd post the result here.
Short version: Ollama behaved well for me, but only after I stopped ignoring batch size.
Happy to post more details if anyone wants them.