+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: PyTorch / JAX / Other Frameworks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=34)
+--- Thread: JAX for inference only (/showthread.php?tid=1151)
Pages:
1
2
JAX for inference only - quietarch - 10-14-2026
Did a bit of testing and thought I'd post the result here.
Short version: MLX behaved well for me, but only after I stopped ignoring quant drift.
Curious if anyone else landed in the same place.
Re: JAX for inference only - plainscope50 - 10-14-2026
Quote:Did a bit of testing and thought I'd post the result here.
that fixed it for me too.
Re: JAX for inference only - vramridge - 10-14-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Re: JAX for inference only - bytestack - 10-15-2026
Quote:Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
My first guess would be this: If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: JAX for inference only - cacheworks - 10-15-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: JAX for inference only - northhall88 - 10-16-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: JAX for inference only - lablab - 10-16-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: JAX for inference only - deepseeked - 10-16-2026
same here on my side.
Re: JAX for inference only - modbench - 10-18-2026
My first guess would be this: On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: JAX for inference only - dockkite69 - 10-18-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?