Help with Audio / Speech with Ollama

[cacheworks]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around SGLang and A6000.

Would rather learn from your scars than collect my own all at once.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

[sparrowkite53]

Need a little more detail here. Are you testing this on Threadripper box or something else?

[cozywire]

Need a little more detail here. Are you testing this on Threadripper box or something else?

Appreciate the detail. The pattern around small local stacks has been pretty similar for me.

[sparrowhall]

Need a little more detail here. Are you testing this on Threadripper box or something else?

A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[vectorfield]

Thanks, that helps. That matches what I saw on dual 3090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[vramridge]

worth checking the driver first.

[audiopoint18]

subscribed, curious where this lands.

[cudapoint]

I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.

[vramport]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.

[signalpeak]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

I'm happy to test one more variant if you want a second data point.