+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Help with Embeddings (/showthread.php?tid=824)
Pages:
1
2
Help with Embeddings - torchdesk - 10-07-2026
Posting this in case it saves someone else a little time.
If you were starting from scratch today for this use case, what would you choose first and what would you ignore?
Would rather learn from your scars than collect my own all at once.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: Help with Embeddings - quantnest - 10-08-2026
My first guess would be this: On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Help with Embeddings - paperport64 - 10-08-2026
Thanks, that helps. The pattern around small local stacks has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Help with Embeddings - edgecoast - 10-08-2026
Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: Help with Embeddings - vectorshed43 - 10-09-2026
I'd start with the simpler stack.
Re: Help with Embeddings - edgecoast - 10-09-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
I'm happy to test one more variant if you want a second data point.
Re: Help with Embeddings - quantnest - 10-10-2026
Thanks, that helps. I ended up in about the same place using PyTorch. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Help with Embeddings - vramscope - 10-10-2026
A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: Help with Embeddings - rasterhall - 10-10-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: Help with Embeddings - vramscope - 10-10-2026
subscribed, curious where this lands.