Help with Embeddings

[quantnest]

Thought I'd post a small recipe because a lot of threads ask similar things.

Maybe useful as a starting point: llama.cpp on dual 3090, keeping embeddings in mind from the start.

Maybe useful as a starting point for newer folks.

[edgecoast]

Thought I'd post a small recipe because a lot of threads ask similar things.

I'd approach it in a boring order. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.