+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Model Merging (https://www.datasculpting.ai/inference/forumdisplay.php?fid=57)
+--- Thread: vector DB that stayed out of the way with vLLM (/showthread.php?tid=1212)
Pages:
1
2
vector DB that stayed out of the way with vLLM - dockember - 10-15-2026
Posting this in case it saves someone else a little time.
If you were starting from scratch today for this use case, what would you choose first and what would you ignore?
Would rather learn from your scars than collect my own all at once.
Re: vector DB that stayed out of the way with vLLM - homelabhenry - 10-17-2026
I'd start with the simpler stack.
Re: vector DB that stayed out of the way with vLLM - thetafield - 10-17-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: vector DB that stayed out of the way with vLLM - orbitbench6 - 10-17-2026
A couple questions before I guess too hard. Are you testing this on 4090 or something else?
Re: vector DB that stayed out of the way with vLLM - mlxgarden - 10-17-2026
that fixed it for me too.
Re: vector DB that stayed out of the way with vLLM - fastscope - 10-17-2026
Quote:that fixed it for me too.
I hit almost the same issue last night.
Re: vector DB that stayed out of the way with vLLM - ggufvault - 10-17-2026
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: vector DB that stayed out of the way with vLLM - stackpilot - 10-17-2026
That matches what I was seeing. The pattern around reranking has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: vector DB that stayed out of the way with vLLM - tensorfield88 - 10-18-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off.
Re: vector DB that stayed out of the way with vLLM - sparrowkite53 - 10-18-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there.