+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: PyTorch / JAX / Other Frameworks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=34)
+--- Thread: JAX for inference only (/showthread.php?tid=1421)
JAX for inference only - tinycontext - 10-17-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about prefill/decode split.
Would rather learn from your scars than collect my own all at once.
Re: JAX for inference only - torchsupply99 - 10-17-2026
Need a little more detail here. Are you testing this on GB200 or something else?
Re: JAX for inference only - cobalttinker - 10-17-2026
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.