+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Model Merging (https://www.datasculpting.ai/inference/forumdisplay.php?fid=57)
+--- Thread: vector DB that stayed out of the way on dual 3090 (/showthread.php?tid=395)
vector DB that stayed out of the way on dual 3090 - nanogarden - 09-21-2026
This keeps coming up in conversation so I figured I'd start a thread.
It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with reranking.
Wondering if this is just where the whole space is heading now.
Re: vector DB that stayed out of the way on dual 3090 - benchplease - 09-22-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal.
Re: vector DB that stayed out of the way on dual 3090 - paperrunner - 09-22-2026
A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece. Turns out 'just one more test' was a lie.
Re: vector DB that stayed out of the way on dual 3090 - vramstream - 09-25-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: vector DB that stayed out of the way on dual 3090 - quietcraft - 09-25-2026
I tested this tonight. On B200 with TensorRT-LLM, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: vector DB that stayed out of the way on dual 3090 - sparrowshed - 09-25-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher.
Re: vector DB that stayed out of the way on dual 3090 - phasespark - 09-25-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: vector DB that stayed out of the way on dual 3090 - tinycontext - 09-26-2026
Did a small comparison this afternoon. On Threadripper box with MLX, I saw basically the same behavior. I expected the model choice to matter more than it did.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.