+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: embedding model you trust on 7900 XTX (/showthread.php?tid=993)
Pages:
1
2
embedding model you trust on 7900 XTX - circuitkite - 10-10-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on mini PC, I ended up with decent performance, but the setup only really made sense once I adjusted around prefix caching.
Curious if anyone else landed in the same place.
Re: embedding model you trust on 7900 XTX - southtinker - 10-11-2026
Quote:Did a bit of testing and thought I'd post the result here.
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: embedding model you trust on 7900 XTX - paperport64 - 10-11-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: embedding model you trust on 7900 XTX - vramghost - 10-12-2026
My first guess would be this: If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: embedding model you trust on 7900 XTX - embershed99 - 10-12-2026
What worked for me was pretty simple. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: embedding model you trust on 7900 XTX - rocmplane - 10-15-2026
Good point. That matches what I saw on Threadripper box. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: embedding model you trust on 7900 XTX - promptpoint - 10-18-2026
Appreciate the detail. I ended up in about the same place using MLX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: embedding model you trust on 7900 XTX - mangohouse - 10-18-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: embedding model you trust on 7900 XTX - cacheline - 10-18-2026
Interesting, I hadn't tried that. I ended up in about the same place using TensorRT-LLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: embedding model you trust on 7900 XTX - quiettoken - 10-18-2026
That matches what I was seeing. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.