desktop apps around local AI with LM Studio

[kernelgrid]

[quote]My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. A l

that fixed it for me too. Turns out 'just one more test' was a lie.

[rocmcrate26]

[quote][quote]My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressu

Interesting, I hadn't tried that. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.