+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Search / Retrieval / Reranking discussion (/showthread.php?tid=884)
Search / Retrieval / Reranking discussion - cudaline - 10-08-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about VRAM headroom.
I'm especially interested in mistakes worth avoiding.
Re: Search / Retrieval / Reranking discussion - sparrowgrid - 10-09-2026
Quote:Posting this in case it saves someone else a little time.
following this because I'm in the same boat.
Re: Search / Retrieval / Reranking discussion - phasestream53 - 10-09-2026
What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.