+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: PyTorch / JAX / Other Frameworks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=34)
+--- Thread: PyTorch compile got faster?: worth it? (/showthread.php?tid=958)
Pages:
1
2
PyTorch compile got faster?: worth it? - kernelmike - 10-09-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once batch size becomes important.
Interested in tradeoffs more than fan loyalty.
This is mostly for local work rather than public serving.
Re: PyTorch compile got faster?: worth it? - vectorfield - 10-09-2026
Quote:Spent some time on this today and wanted to compare notes.
I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: PyTorch compile got faster?: worth it? - quietstack - 10-09-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: PyTorch compile got faster?: worth it? - paperrunner - 10-11-2026
worth checking the driver first.
Re: PyTorch compile got faster?: worth it? - mlxgarden - 10-14-2026
Good point. I don't think your conclusion is far off.
Re: PyTorch compile got faster?: worth it? - amberpatch25 - 10-14-2026
Need a little more detail here. Is this local-only or are you routing anything externally?
Re: PyTorch compile got faster?: worth it? - kernelghost - 10-14-2026
Good point. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: PyTorch compile got faster?: worth it? - torchkite - 10-14-2026
Thanks, that helps. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: PyTorch compile got faster?: worth it? - localtinker - 10-14-2026
Quote:Thanks, that helps. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether t
Appreciate the detail. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: PyTorch compile got faster?: worth it? - macrostack69 - 10-15-2026
My first guess would be this: On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.