+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Image (https://www.datasculpting.ai/inference/forumdisplay.php?fid=69)
+--- Thread: small speech stack for local use: setup advice (/showthread.php?tid=97)
Pages:
1
2
Re: small speech stack for local use: setup advice - llmtrack - 09-10-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: small speech stack for local use: setup advice - deltastation - 09-10-2026
Appreciate the detail. That matches what I saw on B200.
Re: small speech stack for local use: setup advice - bytegarden73 - 09-12-2026
Need a little more detail here. What model, what runtime, and roughly what context size?
Re: small speech stack for local use: setup advice - vramarch - 09-12-2026
Thanks, that helps. The pattern around KV cache pressure has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.