+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Model Merging (https://www.datasculpting.ai/inference/forumdisplay.php?fid=57)
+--- Thread: do you even need a vector DB here on B200 (/showthread.php?tid=1451)
Pages:
1
2
Re: do you even need a vector DB here on B200 - fusedtinker - 10-19-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at long context. Not glamorous advice, but repeatable baselines save a lot of time.
Re: do you even need a vector DB here on B200 - bytepilot - 10-19-2026
My first guess would be this: On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: do you even need a vector DB here on B200 - quantshed - 10-19-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: do you even need a vector DB here on B200 - plainember52 - 10-19-2026
Ran a quick check on my box. On Threadripper box with MLX, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: do you even need a vector DB here on B200 - circuitwire58 - 10-19-2026
Interesting, I hadn't tried that. I ended up in about the same place using PyTorch. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.