+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: what are you all running this week on dual 3090 (/showthread.php?tid=124)
Pages:
1
2
what are you all running this week on dual 3090 - phasetrack - 09-04-2026
I'm trying to sanity-check my setup.
I'm mostly working with Llama in SGLang on 4090. The part I'm trying to understand is prefill/decode split.
Not looking for the perfect answer, mostly what worked in real use.
This is mostly for local work rather than public serving.
Re: what are you all running this week on dual 3090 - plaingarden18 - 09-04-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
Re: what are you all running this week on dual 3090 - marbletrack - 09-04-2026
Quote:What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: what are you all running this week on dual 3090 - plaingarden18 - 09-04-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at batch size.
Re: what are you all running this week on dual 3090 - ggufwire22 - 09-04-2026
Quote:My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at batch size.
subscribed, curious where this lands.
Re: what are you all running this week on dual 3090 - marbletrack - 09-05-2026
that fixed it for me too.
Re: what are you all running this week on dual 3090 - cozygrid12 - 09-05-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: what are you all running this week on dual 3090 - rocmrunner40 - 09-08-2026
that fixed it for me too.
Re: what are you all running this week on dual 3090 - turbogrid11 - 09-08-2026
I hit almost the same issue last night.
Re: what are you all running this week on dual 3090 - tensorkite - 09-08-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.