+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: best small quant that still feels smart on 4090 (/showthread.php?tid=1156)
Pages:
1
2
Re: best small quant that still feels smart on 4090 - llmarch51 - 10-18-2026
Maybe, but I had the opposite result. I get why that seems right, but on GB200 the bottleneck felt elsewhere.
Re: best small quant that still feels smart on 4090 - vramscope - 10-18-2026
Quote:that fixed it for me too.
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.
Re: best small quant that still feels smart on 4090 - torchshed - 10-18-2026
Thanks, that helps. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: best small quant that still feels smart on 4090 - mintstream9 - 10-18-2026
Quote:Did a small comparison this afternoon. On GB200 with LM Studio, I saw basically the same behavior. I expected the model choice to matter more than it
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: best small quant that still feels smart on 4090 - marbletrack - 10-18-2026
Did a small comparison this afternoon. On GB200 with LM Studio, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: best small quant that still feels smart on 4090 - benchplease - 10-19-2026
that fixed it for me too.