10-07-2026, 02:29 AM
I think the bottleneck is somewhere else. I've seen people blame prefill/decode split when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
small models can still be useful