whisper alternatives lately on M3 Max

[visionforge]

Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once tail latency becomes important.

What would you pick if you had to live with it for a few months?

I care more about stability than squeezing the last possible tokens/sec out of it.

[northhall88]

I hit almost the same issue last night. I keep promising myself this is the last rebuild.

[tinycontext]

A couple questions before I guess too hard. Are you testing this on 7900 XTX or something else?

[cobaltworks25]

I don't think that's the whole story. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.

[phasetrack]

I hit almost the same issue last night.

[visionforge]

Thanks, that helps. The pattern around long context has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[bytegarden73]

I think the bottleneck is somewhere else. I've seen people blame reranking when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[vectorfield]

Did a small comparison this afternoon. On 7900 XTX with TensorRT-LLM, I saw basically the same behavior. The main thing that moved was latency, not quality.

[cozykite]

I hit almost the same issue last night.

Did a small comparison this afternoon. On Threadripper box with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.

[visionforge]

I hit almost the same issue last night.