Help with Audio / Speech

[turboscope]

Thought I'd post a small recipe because a lot of threads ask similar things.

Maybe useful as a starting point: vLLM on 5090, keeping quant drift in mind from the start.

Maybe useful as a starting point for newer folks.

[cudapoint]

I hit almost the same issue last night.

[sparrowhall]

What worked for me was pretty simple. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious.

[cudapoint]

A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[sparrowkite53]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?

[benchplease]

Appreciate the detail. The pattern around long context has been pretty similar for me.

[sanescope]

Appreciate the detail. The pattern around long context has been pretty similar for me.

What worked for me was pretty simple. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[bytesupply76]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at router policy. Once you know the plain path works, then add the faster path back in piece by piece.

[vectorfield]

I think the bottleneck is somewhere else. I get why that seems right, but on M3 Max the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[flintriver50]

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.