best small quant that still feels smart on 4090

[llmarch51]

Maybe, but I had the opposite result. I get why that seems right, but on GB200 the bottleneck felt elsewhere.

[vramscope]

that fixed it for me too.

Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.

[torchshed]

Thanks, that helps. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[mintstream9]

Did a small comparison this afternoon. On GB200 with LM Studio, I saw basically the same behavior. I expected the model choice to matter more than it

What exact setup are you running? What model, what runtime, and roughly what context size?

[marbletrack]

Did a small comparison this afternoon. On GB200 with LM Studio, I saw basically the same behavior. I expected the model choice to matter more than it did.

[benchplease]

that fixed it for me too.