QLoRA resource reality check on A6000

[batchpeak]

Need a little more detail here. What model, what runtime, and roughly what context size?

[routearch]

Good point. I ended up in about the same place using LM Studio. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[kernelghost]

Good point. I ended up in about the same place using LM Studio. The biggest change for me was simplifying the stack instead of adding one more optimiz

Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.