+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Structured Output (https://www.datasculpting.ai/inference/forumdisplay.php?fid=63)
+--- Thread: JSON mode across models for Mistral (/showthread.php?tid=651)
Pages:
1
2
JSON mode across models for Mistral - pixelcrate82 - 10-01-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about prefill/decode split.
Simple and boring is a valid answer.
Re: JSON mode across models for Mistral - mangopath4 - 10-01-2026
Thanks, that helps. That matches what I saw on GB200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: JSON mode across models for Mistral - oxideworks59 - 10-02-2026
My first guess would be this: On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: JSON mode across models for Mistral - tensorfield88 - 10-02-2026
Thanks, that helps. I ended up in about the same place using PyTorch.
Re: JSON mode across models for Mistral - llmarch51 - 10-03-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: JSON mode across models for Mistral - cacheplane98 - 10-03-2026
Good point. The pattern around workstation thermals has been pretty similar for me.
Re: JSON mode across models for Mistral - emberquill - 10-04-2026
that fixed it for me too.
Re: JSON mode across models for Mistral - vramstream - 10-04-2026
My first guess would be this: If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: JSON mode across models for Mistral - quietsupply11 - 10-05-2026
Can you fill in a bit more context? Are you testing this on dual 3090 or something else?
Re: JSON mode across models for Mistral - sparrowgrid - 10-06-2026
Quote:Can you fill in a bit more context? Are you testing this on dual 3090 or something else?
I'd start with the simpler stack.