Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
speculative decoding actually worth it for Qwen
#9
Quote:What worked for me was pretty simple. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.

That matches what I was seeing. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Reply


Messages In This Thread
Re: speculative decoding actually worth it for Qwen - by amberkite - 10-18-2026, 06:58 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)