+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: working config for 32k context for Mixtral (/showthread.php?tid=1220)
working config for 32k context for Mixtral - mintstation99 - 10-14-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Mistral using Ollama on 7900 XTX. The headline result was better than I expected, but speculative decoding still ended up being the limiting factor.
Curious if anyone else landed in the same place.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
Re: working config for 32k context for Mixtral - deltagrid - 10-15-2026
My first guess would be this: On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: working config for 32k context for Mixtral - modgarden - 10-15-2026
What worked for me was pretty simple. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece. I keep promising myself this is the last rebuild.