workflow automation with local models

[ggufvault]

I'm trying to sanity-check my setup.

I'm mostly working with Llama in LM Studio on A6000. The part I'm trying to understand is tail latency.

Not looking for the perfect answer, mostly what worked in real use.

[labline]

Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.

[relaynest82]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.