+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Embeddings discussion (/showthread.php?tid=1219)
Embeddings discussion - ggufwire22 - 10-15-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on GB200, or squeeze more out of the stack and deal with the complexity.
Interested in tradeoffs more than fan loyalty.
Re: Embeddings discussion - torchdesk - 10-15-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Embeddings discussion - edgecoast - 10-15-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
Re: Embeddings discussion - tinycontext - 10-18-2026
Good point. I ended up in about the same place using LM Studio. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Embeddings discussion - torchdesk - 10-18-2026
Not sure I agree there. I get why that seems right, but on 4090 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: Embeddings discussion - paperport64 - 10-19-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: Embeddings discussion - edgequill - 10-19-2026
Quote:Good point. I ended up in about the same place using LM Studio. The biggest change for me was simplifying the stack instead of adding one more optimiz
Good point. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Embeddings discussion - vramscope - 10-19-2026
Interesting, I hadn't tried that. I ended up in about the same place using MLX.