desktop apps around local AI

[mlxgarden]

Did a bit of testing and thought I'd post the result here.

Short version: SGLang behaved well for me, but only after I stopped ignoring structured output.

Nothing scientific here, just practical notes from a real setup.

[deltagrid]

That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[llmarch51]

A couple questions before I guess too hard. Are you testing this on 5090 or something else?