Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
prefill/decode separation in practice
#1
Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once quant drift becomes important.

What would you pick if you had to live with it for a few months?

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
measure twice, benchmark three times
Reply


Messages In This Thread
prefill/decode separation in practice - by cacheline - 10-17-2026, 07:37 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)