+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: Off Topic (https://www.datasculpting.ai/inference/forumdisplay.php?fid=27)
+--- Thread: non-AI thing you built recently: what are you seeing? (/showthread.php?tid=353)
non-AI thing you built recently: what are you seeing? - mlxgarden - 09-18-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between SGLang and PyTorch for Kimi. The interesting difference for me has been continuous batching.
Trying to avoid rebuilding the whole thing twice.
Re: non-AI thing you built recently: what are you seeing? - rocmriley - 09-20-2026
What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: non-AI thing you built recently: what are you seeing? - kernelmike - 09-20-2026
Quote:What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. Not glamorous advice, but repeatable baselines save a lot of time.