+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: best small quant that still feels smart on 7900 XTX (/showthread.php?tid=1050)
best small quant that still feels smart on 7900 XTX - benchplease - 10-12-2026
I figured I'd ask here before changing everything.
I'm mostly working with DeepSeek in SGLang on Threadripper box. The part I'm trying to understand is prefill/decode split.
Would appreciate practical advice rather than theory.
Re: best small quant that still feels smart on 7900 XTX - stackarch - 10-12-2026
What exact setup are you running? Are you testing this on GB200 or something else?
Re: best small quant that still feels smart on 7900 XTX - fusedtinker - 10-15-2026
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.