Help with Embeddings on 4090

[vramscope]

Need a little more detail here. Is this local-only or are you routing anything externally?

[vectorshed43]

That matches what I was seeing. I ended up in about the same place using SGLang. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[quantnest]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[paperport64]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these thread

That matches what I was seeing. The pattern around batch size has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[vramscope]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.