4090 vs 5090 for 70B quants

[batchwire]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.

[signaltrack44]

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[rocmriley]

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.