+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: Tensor cores actually helping here on dual 3090 (/showthread.php?tid=582)
Pages:
1
2
Tensor cores actually helping here on dual 3090 - kernelmike - 09-30-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of SGLang and Threadripper box felt surprisingly usable once I tuned around long context.
Maybe useful as a starting point for newer folks.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: Tensor cores actually helping here on dual 3090 - kernelmike - 10-02-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: Tensor cores actually helping here on dual 3090 - phaseport36 - 10-02-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
I'm happy to test one more variant if you want a second data point.
Re: Tensor cores actually helping here on dual 3090 - sparrowbrook - 10-02-2026
same here on my side.
Re: Tensor cores actually helping here on dual 3090 - audioriver - 10-02-2026
I hit almost the same issue last night.
Re: Tensor cores actually helping here on dual 3090 - emberpeak - 10-03-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at NVFP4.
Re: Tensor cores actually helping here on dual 3090 - phaseport36 - 10-06-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Tensor cores actually helping here on dual 3090 - kernelmike - 10-09-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: Tensor cores actually helping here on dual 3090 - phaseport36 - 10-09-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: Tensor cores actually helping here on dual 3090 - dockember - 10-09-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.