driver/framework news roundup: early impressions

[fastlab]

Thought I'd post a small recipe because a lot of threads ask similar things.

Just sharing a setup that ended up working reasonably well: DeepSeek, MLX, A6000, with a lot of attention on INT4.

If someone wants the exact settings I can clean them up and post them.

[bytestack]

I think the bottleneck is somewhere else. I've seen people blame VRAM headroom when the real issue was much more mundane.

[deepseeked]

I tested this tonight. On 5090 with LM Studio, I saw basically the same behavior.

[bytestack]

A couple questions before I guess too hard. Are you testing this on B200 or something else?

[mangowire]

My first guess would be this: On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[deepseeked]

What worked for me was pretty simple. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[thetanest]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there.

[dockriver]

Appreciate the detail. The pattern around batch size has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[audiopoint18]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.