Audio / Speech discussion

[signalpeak]

Did a bit of testing and thought I'd post the result here.

I tried this with Mistral using LM Studio on 7900 XTX. The headline result was better than I expected, but tokens/sec still ended up being the limiting factor.

Nothing scientific here, just practical notes from a real setup.

I expected the biggest difference to come from the model, but the stack around it mattered at least as much.

[cudapoint]

I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[kernelghost]

Thanks, that helps. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[vramforge]

Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?

[sparrowhall]

Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.

[nanogarden]

I hit almost the same issue last night.

[sparrowcraft]

worth checking the driver first.

[cachebench]

that fixed it for me too.

[signalpeak]

that fixed it for me too.

Thanks, that helps. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[sparrowhall]

I'd start with the simpler stack.