VRAM bottleneck on consumer cards for Qwen

[amberhall]

I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[vrambandit]

I'd approach it in a boring order. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[thetarunner]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.

[signaltrack44]

[quote][quote]My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Not glamorou

I don't think that's the whole story. I've seen people blame VRAM headroom when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.

[thetarunner]

Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

I'm happy to test one more variant if you want a second data point.

[amberhall]

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.