10-04-2026, 05:32 PM
Quote:Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
I don't think that's the whole story. I've seen people blame tokens/sec when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.