+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: first LoRA that actually helped on 7900 XTX (/showthread.php?tid=853)
first LoRA that actually helped on 7900 XTX - batchsize - 10-07-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Kimi using llama.cpp on 7900 XTX. The headline result was better than I expected, but prefix caching still ended up being the limiting factor.
Nothing scientific here, just practical notes from a real setup.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: first LoRA that actually helped on 7900 XTX - papermixer4 - 10-07-2026
Quote:Did a bit of testing and thought I'd post the result here.
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there.
Re: first LoRA that actually helped on 7900 XTX - nanobrook - 10-07-2026
subscribed, curious where this lands.
Re: first LoRA that actually helped on 7900 XTX - batchpeak - 10-09-2026
Interesting, I hadn't tried that. That matches what I saw on A6000. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: first LoRA that actually helped on 7900 XTX - cacheline - 10-09-2026
Good point. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: first LoRA that actually helped on 7900 XTX - quiethouse - 10-10-2026
Quote:Good point. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more optimization
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: first LoRA that actually helped on 7900 XTX - thetashed24 - 10-13-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: first LoRA that actually helped on 7900 XTX - cacheline - 10-13-2026
Quote:Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to
I hit almost the same issue last night.
Re: first LoRA that actually helped on 7900 XTX - cachelab - 10-16-2026
My first guess would be this: If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.