09-30-2026, 11:55 AM
Quote:Maybe, but I had the opposite result. I've seen people blame continuous batching when the real issue was much more mundane. Could be a workload differ
Interesting, I hadn't tried that. The pattern around speculative decoding has been pretty similar for me.
post logs or it didn't happen
