+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: best small quant that still feels smart (/showthread.php?tid=912)
best small quant that still feels smart - vrambandit - 10-09-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on A6000, or squeeze more out of the stack and deal with the complexity.
Interested in tradeoffs more than fan loyalty.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: best small quant that still feels smart - cudaline - 10-09-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: best small quant that still feels smart - benchplease - 10-10-2026
following this because I'm in the same boat.
Re: best small quant that still feels smart - northquill - 10-10-2026
same here on my side. I keep promising myself this is the last rebuild.
Re: best small quant that still feels smart - rasterbench - 10-10-2026
Good point. I don't think your conclusion is far off.
Re: best small quant that still feels smart - llmarch51 - 10-10-2026
Good point. That matches what I saw on M3 Max.
Re: best small quant that still feels smart - fastquill74 - 10-10-2026
Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: best small quant that still feels smart - fusedtinker - 10-10-2026
What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: best small quant that still feels smart - benchplease - 10-10-2026
same here on my side.