working config for 32k context for Mixtral

[mintstation99]

Did a bit of testing and thought I'd post the result here.

I tried this with Mistral using Ollama on 7900 XTX. The headline result was better than I expected, but speculative decoding still ended up being the limiting factor.

Curious if anyone else landed in the same place.

I expected the biggest difference to come from the model, but the stack around it mattered at least as much.

[deltagrid]

My first guess would be this: On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[modgarden]

What worked for me was pretty simple. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece. I keep promising myself this is the last rebuild.