+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: 70B still worth the pain for Phi (/showthread.php?tid=404)
Pages:
1
2
70B still worth the pain for Phi - quietsupply11 - 09-21-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on mini PC, or squeeze more out of the stack and deal with the complexity.
Interested in tradeoffs more than fan loyalty.
This is mostly for local work rather than public serving. The logs were somehow calmer than I was.
Re: 70B still worth the pain for Phi - audiorunner74 - 09-21-2026
Good point. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: 70B still worth the pain for Phi - visionforge - 09-23-2026
Quote:Good point. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usab
That matches what I was seeing. That matches what I saw on 5090. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: 70B still worth the pain for Phi - vramstream - 09-23-2026
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: 70B still worth the pain for Phi - mellowsupply - 09-23-2026
worth checking the driver first.
Re: 70B still worth the pain for Phi - batchshed - 09-23-2026
That matches what I was seeing. I don't think your conclusion is far off.
Re: 70B still worth the pain for Phi - localtinker - 09-23-2026
A couple things I'd check first. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: 70B still worth the pain for Phi - deltastation - 09-24-2026
That matches what I was seeing. I don't think your conclusion is far off.
Re: 70B still worth the pain for Phi - plaingarden18 - 09-25-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: 70B still worth the pain for Phi - kernelmike - 09-28-2026
Thanks, that helps. That matches what I saw on Threadripper box. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.