+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Audio / Speech (https://www.datasculpting.ai/inference/forumdisplay.php?fid=71)
+--- Thread: Help with Audio / Speech (/showthread.php?tid=419)
Help with Audio / Speech - thetarunner - 09-23-2026
Maybe I'm missing something obvious.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling batch size.
If you've tested something similar, what did you keep and what did you stop bothering with?
Re: Help with Audio / Speech - quantyard - 09-24-2026
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet. Turns out 'just one more test' was a lie.
Re: Help with Audio / Speech - batchpeak - 09-27-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: Help with Audio / Speech - vectorfield - 09-28-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at router policy. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Help with Audio / Speech - quietarch - 09-29-2026
I'd approach it in a boring order. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: Help with Audio / Speech - bytesupply76 - 09-29-2026
Quote:I'd approach it in a boring order. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives.
A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Help with Audio / Speech - signalpeak - 09-29-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: Help with Audio / Speech - flintriver50 - 10-02-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split.
Re: Help with Audio / Speech - sparrowcraft - 10-02-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off.