QLoRA resource reality check on Threadripper box

[quiethouse]

Ran a quick check on my box. On A6000 with llama.cpp, I saw basically the same behavior. The surprise was that memory pressure showed up before throug

Appreciate the detail. I ended up in about the same place using llama.cpp. The biggest change for me was simplifying the stack instead of adding one more optimization layer.