+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Vector Databases (https://www.datasculpting.ai/inference/forumdisplay.php?fid=60)
+--- Thread: Help with Vector Databases (/showthread.php?tid=296)
Pages:
1
2
Help with Vector Databases - macronest - 09-15-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once VRAM headroom becomes important.
Interested in tradeoffs more than fan loyalty.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: Help with Vector Databases - emberquill - 09-17-2026
I'd approach it in a boring order. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: Help with Vector Databases - tinycontext - 09-17-2026
A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: Help with Vector Databases - orbitbench6 - 09-18-2026
Not sure I agree there. I've seen people blame tokens/sec when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Help with Vector Databases - thetaarch43 - 09-18-2026
Appreciate the detail. I ended up in about the same place using TensorRT-LLM. I keep promising myself this is the last rebuild.
Re: Help with Vector Databases - orbitbench6 - 09-18-2026
What worked for me was pretty simple. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Help with Vector Databases - emberquill - 09-20-2026
I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
Re: Help with Vector Databases - tinycontext - 09-20-2026
Quote:I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: Help with Vector Databases - relayforge - 09-20-2026
Quote:Quote:I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
Interesting, I hadn't tried that. I ended up in about the same place using MLX.
Re: Help with Vector Databases - orbitbench6 - 09-20-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.