+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Audio / Speech (https://www.datasculpting.ai/inference/forumdisplay.php?fid=71)
+--- Thread: Audio / Speech discussion: worth it? (/showthread.php?tid=551)
Audio / Speech discussion: worth it? - bytesupply76 - 09-28-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between SGLang and MLX for Mixtral. The interesting difference for me has been speculative decoding.
Interested in tradeoffs more than fan loyalty.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: Audio / Speech discussion: worth it? - localpatch78 - 09-28-2026
What worked for me was pretty simple. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Audio / Speech discussion: worth it? - sparrowhall - 09-28-2026
My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious.