Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
whisper alternatives lately for DeepSeek
#1
Thought I'd post a small recipe because a lot of threads ask similar things.

Maybe useful as a starting point: MLX on dual 3090, keeping continuous batching in mind from the start.

Feel free to tear it apart if you see something dumb.

The part that surprised me was how much the boring details mattered more than the model choice.
Reply
#2
My first guess would be this: If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Reply
#3
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)