+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Embeddings discussion for Phi (/showthread.php?tid=145)
Embeddings discussion for Phi - rocmmixer - 09-05-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of TensorRT-LLM and A6000 felt surprisingly usable once I tuned around MoE serving.
Maybe useful as a starting point for newer folks.
Re: Embeddings discussion for Phi - dockriver - 09-06-2026
I hit almost the same issue last night.
Re: Embeddings discussion for Phi - nanopatch - 09-06-2026
that fixed it for me too.
Re: Embeddings discussion for Phi - quantnest - 09-06-2026
Interesting, I hadn't tried that. That matches what I saw on mini PC. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Embeddings discussion for Phi - oxidemixer - 09-07-2026
same here on my side.
Re: Embeddings discussion for Phi - paperport64 - 09-07-2026
Quote:same here on my side.
Thanks, that helps. The pattern around router policy has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Embeddings discussion for Phi - torchsmith25 - 09-09-2026
Interesting, I hadn't tried that. I ended up in about the same place using LM Studio.
Re: Embeddings discussion for Phi - paperport64 - 09-09-2026
A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.