LM Studio surprisingly handy on 5090

[edgeriver]

This keeps coming up in conversation so I figured I'd start a thread.

It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with continuous batching.

Curious whether that lines up with what other people are seeing.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

[amberhall]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.

[pixelspark]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before thr

Thanks, that helps. The pattern around tool calling has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[ggufwire22]

I'd start with the simpler stack.

[vectorcrate98]

Can you fill in a bit more context? Is this local-only or are you routing anything externally?

[dockridge]

I'd start with the simpler stack.

[prompttinker44]

I'd start with the simpler stack.

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[mellowwire98]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at structured output.

[dockridge]

Appreciate the detail. The pattern around continuous batching has been pretty similar for me.

[cacheworks]

Thanks, that helps. The pattern around continuous batching has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.