+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: where open models feel surprisingly good for Phi (/showthread.php?tid=1200)
where open models feel surprisingly good for Phi - amberhall - 10-14-2026
Has anyone else run into this?
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling prefix caching.
Would appreciate practical advice rather than theory.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: where open models feel surprisingly good for Phi - bytepilot - 10-16-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: where open models feel surprisingly good for Phi - vrampilot - 10-18-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.