+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Vector Databases (https://www.datasculpting.ai/inference/forumdisplay.php?fid=60)
+--- Thread: Vector Databases discussion: anyone testing this? (/showthread.php?tid=1096)
Pages:
1
2
Re: Vector Databases discussion: anyone testing this? - southspark - 10-18-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: Vector Databases discussion: anyone testing this? - dockridge - 10-18-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: Vector Databases discussion: anyone testing this? - byteyard - 10-18-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure.
Re: Vector Databases discussion: anyone testing this? - routeforge64 - 10-19-2026
What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.