+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: prefill bottlenecks on longer prompts for Mistral (/showthread.php?tid=289)
Pages:
1
2
Re: prefill bottlenecks on longer prompts for Mistral - bytestack - 09-25-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.
Re: prefill bottlenecks on longer prompts for Mistral - cozyarch - 09-26-2026
following this because I'm in the same boat.
Re: prefill bottlenecks on longer prompts for Mistral - rocmhouse90 - 09-26-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: prefill bottlenecks on longer prompts for Mistral - thetanest - 09-26-2026
Quote:I tested this tonight. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
following this because I'm in the same boat.