+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: where open models feel surprisingly good for Phi (/showthread.php?tid=1149)
Pages:
1
2
where open models feel surprisingly good for Phi - thetapath - 10-14-2026
I'm trying to sanity-check my setup.
I'm mostly working with FLUX in Ollama on B200. The part I'm trying to understand is KV cache pressure.
Not looking for the perfect answer, mostly what worked in real use.
Re: where open models feel surprisingly good for Phi - mellowwire98 - 10-14-2026
My first guess would be this: On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: where open models feel surprisingly good for Phi - macrostack69 - 10-17-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: where open models feel surprisingly good for Phi - rocmhouse90 - 10-17-2026
Good point. The pattern around token cost has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: where open models feel surprisingly good for Phi - promptpoint - 10-17-2026
That matches what I was seeing. I ended up in about the same place using LM Studio. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: where open models feel surprisingly good for Phi - sanepeak48 - 10-17-2026
What exact setup are you running? Are you testing this on 7900 XTX or something else?
Re: where open models feel surprisingly good for Phi - embershed99 - 10-17-2026
Did a small comparison this afternoon. On M3 Max with SGLang, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: where open models feel surprisingly good for Phi - batchsize - 10-18-2026
Thanks, that helps. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: where open models feel surprisingly good for Phi - kernelarch - 10-18-2026
I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: where open models feel surprisingly good for Phi - paperrunner - 10-18-2026
A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.