+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: QLoRA resource reality check with SGLang (/showthread.php?tid=409)
QLoRA resource reality check with SGLang - deepseeked - 09-22-2026
I'm trying to sanity-check my setup.
My current setup is SGLang with A6000, and I keep running into long context.
If you've tested something similar, what did you keep and what did you stop bothering with?
Re: QLoRA resource reality check with SGLang - batchsize - 09-23-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
I'm happy to test one more variant if you want a second data point.
Re: QLoRA resource reality check with SGLang - batchpeak - 09-23-2026
My first guess would be this: On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: QLoRA resource reality check with SGLang - batchsize - 09-23-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at HBM constraints. Once you know the plain path works, then add the faster path back in piece by piece.
Re: QLoRA resource reality check with SGLang - benchplease - 09-24-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off.
Re: QLoRA resource reality check with SGLang - routearch - 09-24-2026
Quote:Interesting, I hadn't tried that. I don't think your conclusion is far off.
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: QLoRA resource reality check with SGLang - tensorfield88 - 09-24-2026
Quote:Quote:Interesting, I hadn't tried that. I don't think your conclusion is far off.
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.