+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Image (https://www.datasculpting.ai/inference/forumdisplay.php?fid=69)
+--- Thread: small speech stack for local use with LM Studio (/showthread.php?tid=294)
small speech stack for local use with LM Studio - cobaltspark - 09-15-2026
Posting this in case it saves someone else a little time.
If you were starting from scratch today for this use case, what would you choose first and what would you ignore?
Simple and boring is a valid answer.
I care more about stability than squeezing the last possible tokens/sec out of it.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
Re: small speech stack for local use with LM Studio - quiethouse - 09-15-2026
Maybe, but I had the opposite result. I've seen people blame continuous batching when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: small speech stack for local use with LM Studio - llmtrack - 09-15-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: small speech stack for local use with LM Studio - kernelpeak - 09-17-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher.
Re: small speech stack for local use with LM Studio - tensorworks - 09-17-2026
My first guess would be this: If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: small speech stack for local use with LM Studio - localcraft17 - 09-17-2026
I'd start with the simpler stack.