+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Performance Tuning (https://www.datasculpting.ai/inference/forumdisplay.php?fid=42)
+--- Thread: disaggregated inference thoughts on dual 3090 (/showthread.php?tid=838)
disaggregated inference thoughts on dual 3090 - prompttinker44 - 10-07-2026
Did a bit of testing and thought I'd post the result here.
Short version: LM Studio behaved well for me, but only after I stopped ignoring structured output.
Nothing scientific here, just practical notes from a real setup.
Re: disaggregated inference thoughts on dual 3090 - stackarch - 10-08-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: disaggregated inference thoughts on dual 3090 - rocmpath - 10-09-2026
I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: disaggregated inference thoughts on dual 3090 - kernelmike - 10-09-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: disaggregated inference thoughts on dual 3090 - tensorfield88 - 10-09-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: disaggregated inference thoughts on dual 3090 - sparrowghost - 10-11-2026
Thanks, that helps. That matches what I saw on dual 3090. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: disaggregated inference thoughts on dual 3090 - vramport - 10-11-2026
that fixed it for me too.
Re: disaggregated inference thoughts on dual 3090 - plainline - 10-11-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: disaggregated inference thoughts on dual 3090 - kernelmike - 10-11-2026
Good point. The pattern around small local stacks has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: disaggregated inference thoughts on dual 3090 - signalcraft14 - 10-13-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.