+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Help with Embeddings for Phi (/showthread.php?tid=1013)
Help with Embeddings for Phi - vectorshed43 - 10-11-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around LM Studio and 7900 XTX.
Would rather learn from your scars than collect my own all at once.
Re: Help with Embeddings for Phi - torchdesk - 10-11-2026
What worked for me was pretty simple. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Help with Embeddings for Phi - edgecoast - 10-11-2026
That matches what I was seeing. The pattern around batch size has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Help with Embeddings for Phi - quantnest - 10-13-2026
My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Help with Embeddings for Phi - batchcraft - 10-16-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: Help with Embeddings for Phi - edgequill - 10-17-2026
Thanks, that helps. The pattern around consumer GPUs has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Help with Embeddings for Phi - rasterhall - 10-17-2026
I don't think that's the whole story. I get why that seems right, but on Threadripper box the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
Re: Help with Embeddings for Phi - paperport64 - 10-19-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: Help with Embeddings for Phi - ggufwire22 - 10-19-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Help with Embeddings for Phi - ggufwire22 - 10-19-2026
Need a little more detail here. What model, what runtime, and roughly what context size?