+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Vector Databases (https://www.datasculpting.ai/inference/forumdisplay.php?fid=60)
+--- Thread: Vector Databases discussion: what are you seeing? (/showthread.php?tid=514)
Vector Databases discussion: what are you seeing? - tinycontext - 09-27-2026
Has anyone else run into this?
My current setup is llama.cpp with 4090, and I keep running into consumer GPUs.
If you've tested something similar, what did you keep and what did you stop bothering with?
I care more about stability than squeezing the last possible tokens/sec out of it.
The part that surprised me was how much the boring details mattered more than the model choice.
Re: Vector Databases discussion: what are you seeing? - syshouse - 09-27-2026
Quote:Has anyone else run into this?
I hit almost the same issue last night.
Re: Vector Databases discussion: what are you seeing? - deepseeked - 09-27-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?