+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Embeddings (https://www.datasculpting.ai/inference/forumdisplay.php?fid=59)
+--- Thread: Help with Embeddings (/showthread.php?tid=1189)
Help with Embeddings - quantnest - 10-14-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate workstation thermals until they actually try to run this stuff for more than one afternoon.
Curious whether that lines up with what other people are seeing.
Re: Help with Embeddings - ggufwire22 - 10-15-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: Help with Embeddings - edgecoast - 10-18-2026
Quote:Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pr
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Help with Embeddings - quantnest - 10-18-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Help with Embeddings - vramscope - 10-19-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: Help with Embeddings - vectorshed43 - 10-19-2026
Thanks, that helps. The pattern around KV cache pressure has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Help with Embeddings - torchsupply99 - 10-19-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Help with Embeddings - edgecoast - 10-19-2026
A couple questions before I guess too hard. Are you testing this on mini PC or something else?
Re: Help with Embeddings - ggufwire22 - 10-19-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: Help with Embeddings - torchdesk - 10-19-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?