Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
prefill/decode separation in practice with PyTorch
#1
Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on M3 Max, or squeeze more out of the stack and deal with the complexity.

Interested in tradeoffs more than fan loyalty.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
learning in public
Reply


Messages In This Thread
prefill/decode separation in practice with PyTorch - by thetashed24 - 08-24-2026, 07:29 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)