+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: desktop apps around local AI (/showthread.php?tid=462)
desktop apps around local AI - mlxgarden - 09-24-2026
Did a bit of testing and thought I'd post the result here.
Short version: SGLang behaved well for me, but only after I stopped ignoring structured output.
Nothing scientific here, just practical notes from a real setup.
Re: desktop apps around local AI - deltagrid - 09-25-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: desktop apps around local AI - llmarch51 - 09-25-2026
A couple questions before I guess too hard. Are you testing this on 5090 or something else?