+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: working config for 32k context on B200 (/showthread.php?tid=1143)
working config for 32k context on B200 - deltagrid - 10-13-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Mistral, TensorRT-LLM, GB200, with a lot of attention on INT4.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
Re: working config for 32k context on B200 - rocmcrate26 - 10-15-2026
Thanks, that helps. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.