+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: embedding drift issues for Phi (/showthread.php?tid=1196)
embedding drift issues for Phi - benchplease - 10-13-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around llama.cpp and B200.
Would rather learn from your scars than collect my own all at once.
Re: embedding drift issues for Phi - vramridge - 10-13-2026
Appreciate the detail. The pattern around MoE serving has been pretty similar for me.
Re: embedding drift issues for Phi - visionyard23 - 10-14-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: embedding drift issues for Phi - macrostack69 - 10-14-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: embedding drift issues for Phi - rocmcrate26 - 10-15-2026
Appreciate the detail. That matches what I saw on M3 Max. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: embedding drift issues for Phi - latentgarden - 10-15-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: embedding drift issues for Phi - thetapath - 10-15-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.
Re: embedding drift issues for Phi - thetashed24 - 10-18-2026
Quote:Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.
following this because I'm in the same boat.
Re: embedding drift issues for Phi - amberpatch25 - 10-18-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.