+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Structured Output (https://www.datasculpting.ai/inference/forumdisplay.php?fid=63)
+--- Thread: how strict do you make schemas for Mistral (/showthread.php?tid=1183)
Pages:
1
2
Re: how strict do you make schemas for Mistral - tensorworks - 10-18-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: how strict do you make schemas for Mistral - visionforge - 10-18-2026
Quote:Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at batch size. Not glamorous advice, but repeatable baselines save a lot of time.
Re: how strict do you make schemas for Mistral - tinycontext - 10-19-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: how strict do you make schemas for Mistral - quiettoken - 10-19-2026
Quote:Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency,
Thanks, that helps. The pattern around KV cache pressure has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: how strict do you make schemas for Mistral - cudacrate - 10-19-2026
That matches what I was seeing. The pattern around prefix caching has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: how strict do you make schemas for Mistral - macrolab - 10-19-2026
Quote:Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Wort
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed. At this point the real bottleneck is my patience.