+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: QLoRA resource reality check on Threadripper box (/showthread.php?tid=457)
Pages:
1
2
Re: QLoRA resource reality check on Threadripper box - quiethouse - 09-27-2026
Quote:Ran a quick check on my box. On A6000 with llama.cpp, I saw basically the same behavior. The surprise was that memory pressure showed up before throug
Appreciate the detail. I ended up in about the same place using llama.cpp. The biggest change for me was simplifying the stack instead of adding one more optimization layer.