+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: first LoRA that actually helped (/showthread.php?tid=1340)
first LoRA that actually helped - benchplease - 10-16-2026
I figured I'd ask here before changing everything.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling MoE serving.
Would appreciate practical advice rather than theory.
Re: first LoRA that actually helped - benchplease - 10-18-2026
Maybe, but I had the opposite result. I've seen people blame VRAM headroom when the real issue was much more mundane.
Re: first LoRA that actually helped - batchsize - 10-18-2026
Quote:Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Thanks, that helps. That matches what I saw on 7900 XTX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: first LoRA that actually helped - localtinker - 10-18-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: first LoRA that actually helped - cacheline - 10-19-2026
subscribed, curious where this lands.
Re: first LoRA that actually helped - kernelghost - 10-19-2026
Thanks, that helps. I ended up in about the same place using MLX. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: first LoRA that actually helped - cacheline - 10-19-2026
Quote:Quote:Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. Once you know the plain path works, then add the faster path back in piece by piece.
Re: first LoRA that actually helped - batchsize - 10-19-2026
Quote:Maybe, but I had the opposite result. I've seen people blame VRAM headroom when the real issue was much more mundane.
Maybe, but I had the opposite result. I get why that seems right, but on GB200 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: first LoRA that actually helped - benchplease - 10-19-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.