+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Help with Embeddings on 4090 (/showthread.php?tid=600)
Pages:
1
2
Re: Help with Embeddings on 4090 - vramscope - 10-04-2026
Need a little more detail here. Is this local-only or are you routing anything externally?
Re: Help with Embeddings on 4090 - vectorshed43 - 10-04-2026
That matches what I was seeing. I ended up in about the same place using SGLang. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Help with Embeddings on 4090 - quantnest - 10-04-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Help with Embeddings on 4090 - paperport64 - 10-05-2026
Quote:I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these thread
That matches what I was seeing. The pattern around batch size has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Help with Embeddings on 4090 - vramscope - 10-05-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.