working config for 32k context on B200

[deltagrid]

Thought I'd post a small recipe because a lot of threads ask similar things.

Just sharing a setup that ended up working reasonably well: Mistral, TensorRT-LLM, GB200, with a lot of attention on INT4.

Feel free to tear it apart if you see something dumb.

This is mostly for local work rather than public serving.

[rocmcrate26]

Thanks, that helps. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.