+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: CPUs / NPUs / Accelerators (https://www.datasculpting.ai/inference/forumdisplay.php?fid=30)
+--- Thread: Any point in NPUs yet on dual 3090 (/showthread.php?tid=1094)
Pages:
1
2
Any point in NPUs yet on dual 3090 - thetashed24 - 10-12-2026
I'm trying to sanity-check my setup.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling tokens/sec.
Would appreciate practical advice rather than theory.
Re: Any point in NPUs yet on dual 3090 - batchwire - 10-13-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal.
Re: Any point in NPUs yet on dual 3090 - cudahouse - 10-14-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Any point in NPUs yet on dual 3090 - bytehall - 10-14-2026
Interesting, I hadn't tried that. That matches what I saw on B200. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Any point in NPUs yet on dual 3090 - cudahouse - 10-14-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at VRAM headroom. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Any point in NPUs yet on dual 3090 - latentscope35 - 10-15-2026
worth checking the driver first.
Re: Any point in NPUs yet on dual 3090 - batchsize - 10-16-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
I'm happy to test one more variant if you want a second data point.
Re: Any point in NPUs yet on dual 3090 - batchyard - 10-17-2026
My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Any point in NPUs yet on dual 3090 - cudacrate - 10-17-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: Any point in NPUs yet on dual 3090 - cacheline - 10-18-2026
Quote:Appreciate the detail. The pattern around embeddings has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption
Interesting, I hadn't tried that. I ended up in about the same place using PyTorch. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.