+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: embedding drift issues for Llama (/showthread.php?tid=267)
embedding drift issues for Llama - cachelab - 09-13-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of MLX and A6000 felt surprisingly usable once I tuned around embeddings.
Maybe useful as a starting point for newer folks.
Re: embedding drift issues for Llama - macropoint - 09-15-2026
Appreciate the detail. I ended up in about the same place using vLLM.
Re: embedding drift issues for Llama - phaseport36 - 09-15-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
If you post your exact settings, people here can probably narrow it down pretty fast. At this point the real bottleneck is my patience.
Re: embedding drift issues for Llama - vramarch - 09-15-2026
Good point. I ended up in about the same place using SGLang. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: embedding drift issues for Llama - dockbrook21 - 09-16-2026
Not sure I agree there. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
Re: embedding drift issues for Llama - kernelmike - 09-16-2026
following this because I'm in the same boat.
Re: embedding drift issues for Llama - deepseeked - 09-16-2026
Thanks, that helps. I ended up in about the same place using Ollama.
Re: embedding drift issues for Llama - signaltrack44 - 09-16-2026
Thanks, that helps. That matches what I saw on B200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.