Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
speculative decoding actually worth it with PyTorch
#6
Good point. The pattern around reranking has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
measure twice, benchmark three times
Reply


Messages In This Thread
Re: speculative decoding actually worth it with PyTorch - by cacheline - 08-30-2026, 05:31 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)