+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: PyTorch / JAX / Other Frameworks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=34)
+--- Thread: PyTorch compile got faster? on dual 3090 (/showthread.php?tid=1480)
Re: PyTorch compile got faster? on dual 3090 - phaseport36 - 10-18-2026
subscribed, curious where this lands.
I'm happy to test one more variant if you want a second data point.
PyTorch compile got faster? on dual 3090 - byteyard - 10-18-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once structured output becomes important.
Trying to avoid rebuilding the whole thing twice.
Re: PyTorch compile got faster? on dual 3090 - vrambandit - 10-19-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: PyTorch compile got faster? on dual 3090 - circuitwire58 - 10-19-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: PyTorch compile got faster? on dual 3090 - batchscope72 - 10-19-2026
Thanks, that helps. I ended up in about the same place using LM Studio.
Re: PyTorch compile got faster? on dual 3090 - dockember - 10-19-2026
Not sure I agree there. I've seen people blame tokens/sec when the real issue was much more mundane.
Re: PyTorch compile got faster? on dual 3090 - mintstream9 - 10-19-2026
That matches what I was seeing. The pattern around quant drift has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.