+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Evals (https://www.datasculpting.ai/inference/forumdisplay.php?fid=66)
+--- Thread: human evals on small teams for Phi (/showthread.php?tid=1449)
Pages:
1
2
Re: human evals on small teams for Phi - vectorcrate98 - 10-18-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.
Re: human evals on small teams for Phi - kernelarch - 10-18-2026
A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: human evals on small teams for Phi - oxideworks59 - 10-18-2026
worth checking the driver first.
Re: human evals on small teams for Phi - promptpoint - 10-19-2026
that fixed it for me too.
Re: human evals on small teams for Phi - oxideworks59 - 10-19-2026
Quote:Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: human evals on small teams for Phi - vrambandit - 10-19-2026
subscribed, curious where this lands. I keep promising myself this is the last rebuild.
Re: human evals on small teams for Phi - rocmriley - 10-19-2026
Not sure I agree there. I've seen people blame speculative decoding when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
human evals on small teams for Phi - kernelgrid - 10-19-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like continuous batching decides everything downstream.
Maybe obvious to some of you, but it clicked for me this week.
Re: human evals on small teams for Phi - promptpoint - 10-19-2026
Appreciate the detail. That matches what I saw on 5090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: human evals on small teams for Phi - bytepilot - 10-19-2026
Good point. I ended up in about the same place using LM Studio. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.