+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: QLoRA resource reality check on Threadripper box (/showthread.php?tid=457)
Pages:
1
2
QLoRA resource reality check on Threadripper box - turboyard94 - 09-24-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around SGLang and A6000.
Simple and boring is a valid answer.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: QLoRA resource reality check on Threadripper box - cacheline - 09-25-2026
Quote:Posting this in case it saves someone else a little time.
Did a small comparison this afternoon. On dual 3090 with TensorRT-LLM, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: QLoRA resource reality check on Threadripper box - deltastation - 09-25-2026
I'd approach it in a boring order. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: QLoRA resource reality check on Threadripper box - batchsize - 09-25-2026
worth checking the driver first.
Re: QLoRA resource reality check on Threadripper box - cacheline - 09-25-2026
subscribed, curious where this lands.
Re: QLoRA resource reality check on Threadripper box - batchsize - 09-25-2026
My first guess would be this: On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: QLoRA resource reality check on Threadripper box - sanevault - 09-25-2026
following this because I'm in the same boat.
Re: QLoRA resource reality check on Threadripper box - deltastation - 09-25-2026
I'd approach it in a boring order. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. I keep promising myself this is the last rebuild.
Re: QLoRA resource reality check on Threadripper box - nanobrook - 09-25-2026
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: QLoRA resource reality check on Threadripper box - kernelghost - 09-26-2026
Ran a quick check on my box. On A6000 with llama.cpp, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.