+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: industry move that matters for local users (/showthread.php?tid=1387)
Pages:
1
2
Re: industry move that matters for local users - bytelab - 10-18-2026
I think the bottleneck is somewhere else. I get why that seems right, but on B200 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: industry move that matters for local users - deepseeked - 10-19-2026
What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: industry move that matters for local users - bytestack - 10-19-2026
Good point. I don't think your conclusion is far off.
Re: industry move that matters for local users - dockriver - 10-19-2026
I tested this tonight. On 4090 with MLX, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: industry move that matters for local users - thetashed24 - 10-19-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: industry move that matters for local users - quietsupply11 - 10-19-2026
Maybe, but I had the opposite result. I've seen people blame small local stacks when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.