Help with Audio / Speech

[signalcraft14]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around SGLang and Threadripper box.

Would rather learn from your scars than collect my own all at once.

[sparrowkite53]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal.

[vramforge]

Ran a quick check on my box. On 5090 with llama.cpp, I saw basically the same behavior. The main thing that moved was latency, not quality.

[batchpeak]

A couple questions before I guess too hard. Are you testing this on mini PC or something else?

[cudapoint]

A couple questions before I guess too hard. Are you testing this on mini PC or something else?

Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[flintriver50]

What worked for me was pretty simple. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[vramport]

worth checking the driver first.

[vramridge]

I'd start with the simpler stack.

[sparrowhall]

Appreciate the detail. The pattern around prefix caching has been pretty similar for me.

[cozywire]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?