+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Model Merging (https://www.datasculpting.ai/inference/forumdisplay.php?fid=57)
+--- Thread: do you even need a vector DB here for Mistral (/showthread.php?tid=1246)
Pages:
1
2
Re: do you even need a vector DB here for Mistral - vectorshed43 - 10-18-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. Once you know the plain path works, then add the faster path back in piece by piece.
Re: do you even need a vector DB here for Mistral - dockember - 10-19-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal.
Re: do you even need a vector DB here for Mistral - routerunner - 10-19-2026
that fixed it for me too.
Re: do you even need a vector DB here for Mistral - southspark - 10-19-2026
A couple things I'd check first. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.