+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Search / Retrieval / Reranking discussion (/showthread.php?tid=875)
Pages:
1
2
Re: Search / Retrieval / Reranking discussion - batchember - 10-13-2026
Interesting, I hadn't tried that. I ended up in about the same place using TensorRT-LLM.
Re: Search / Retrieval / Reranking discussion - kernelarch - 10-13-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: Search / Retrieval / Reranking discussion - papermixer4 - 10-13-2026
What worked for me was pretty simple. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: Search / Retrieval / Reranking discussion - vramscope - 10-14-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I keep promising myself this is the last rebuild.
Re: Search / Retrieval / Reranking discussion - kernelhouse70 - 10-16-2026
A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.