small speech stack for local use: setup advice

[macropoint]

I hit almost the same issue last night.

[cobaltspark]

Interesting, I hadn't tried that. The pattern around speculative decoding has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[visionforge]

Need a little more detail here. Is this local-only or are you routing anything externally?

[amberpatch25]

Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.

[bytegarden73]

Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.

I'd approach it in a boring order. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.