10-19-2026, 02:40 AM
Quote:[quote]Thanks, that helps. The pattern around structured output has been pretty similar for me. I got better results once I stopped chasing one metric
I think the bottleneck is somewhere else. I've seen people blame prefix caching when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.
measure twice, benchmark three times