Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
prefix caching on repeated workflows for Gemma
#1
Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once MoE serving becomes important.

What would you pick if you had to live with it for a few months?
watching release notes so you don't have to
Reply


Messages In This Thread
prefix caching on repeated workflows for Gemma - by deepseeked - 10-13-2026, 08:29 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)