Best workstation layout for local models with LM Studio

[latentpeak22]

Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once small local stacks becomes important.

Trying to avoid rebuilding the whole thing twice.

[vectorport]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.

[plainscope50]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, no

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[tensorworks]

I hit almost the same issue last night.

[labnest]

I hit almost the same issue last night.

Good point. The pattern around continuous batching has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

If you post your exact settings, people here can probably narrow it down pretty fast.

[phasenode]

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.

[homelabhenry]

I'd approach it in a boring order. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.

[stackpilot]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.