when TensorRT-LLM clearly wins

[cachebench]

Spent some time on this today and wanted to compare notes.

I've gone back and forth between SGLang and llama.cpp for Mixtral. The interesting difference for me has been long context.

What would you pick if you had to live with it for a few months?

[kernelmike]

Spent some time on this today and wanted to compare notes.

Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.

[quietsupply11]

A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.