+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Help with Embeddings (/showthread.php?tid=533)
Help with Embeddings - papermixer4 - 09-28-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of Ollama and 4090 felt surprisingly usable once I tuned around long context.
Maybe useful as a starting point for newer folks.
Re: Help with Embeddings - vramscope - 09-28-2026
What exact setup are you running? Are you testing this on B200 or something else?
Re: Help with Embeddings - rasterhall - 09-28-2026
I'd start with the simpler stack.
Re: Help with Embeddings - ggufwire22 - 09-28-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tail latency. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Help with Embeddings - lablab - 09-28-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Help with Embeddings - edgecoast - 10-01-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: Help with Embeddings - vectorshed43 - 10-02-2026
Thanks, that helps. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.