+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Help with Embeddings: what are you seeing? (/showthread.php?tid=266)
Pages:
1
2
Help with Embeddings: what are you seeing? - edgecoast - 09-13-2026
Did a bit of testing and thought I'd post the result here.
I tried this with DeepSeek using SGLang on 7900 XTX. The headline result was better than I expected, but VRAM headroom still ended up being the limiting factor.
Curious if anyone else landed in the same place.
The part that surprised me was how much the boring details mattered more than the model choice.
Re: Help with Embeddings: what are you seeing? - vectorshed43 - 09-15-2026
A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Help with Embeddings: what are you seeing? - paperport64 - 09-18-2026
worth checking the driver first.
Re: Help with Embeddings: what are you seeing? - llmtrack26 - 09-18-2026
Thanks, that helps. I ended up in about the same place using LM Studio. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Help with Embeddings: what are you seeing? - vramscope - 09-18-2026
Thanks, that helps. That matches what I saw on 4090.
Re: Help with Embeddings: what are you seeing? - edgecoast - 09-18-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. Not glamorous advice, but repeatable baselines save a lot of time.
I'm happy to test one more variant if you want a second data point.
Re: Help with Embeddings: what are you seeing? - paperport64 - 09-18-2026
Tried to reproduce it just now. On 5090 with LM Studio, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: Help with Embeddings: what are you seeing? - kernelhouse70 - 09-19-2026
same here on my side.
Re: Help with Embeddings: what are you seeing? - torchdesk - 09-19-2026
What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Help with Embeddings: what are you seeing? - edgecoast - 09-19-2026
I think the bottleneck is somewhere else. I get why that seems right, but on 7900 XTX the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.