Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
speculative decoding in llama.cpp
#13
I think the bottleneck is somewhere else. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
Reply


Messages In This Thread
speculative decoding in llama.cpp - by mangoquill - 09-14-2026, 03:43 PM
Re: speculative decoding in llama.cpp - by bytesupply76 - 09-26-2026, 06:09 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)