+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: where reranking mattered: worth it? (/showthread.php?tid=550)
Pages:
1
2
where reranking mattered: worth it? - tinycontext - 09-29-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around Ollama and A6000.
Would rather learn from your scars than collect my own all at once.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Re: where reranking mattered: worth it? - homelabhenry - 09-29-2026
That matches what I was seeing. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: where reranking mattered: worth it? - modbench - 09-29-2026
Quote:That matches what I was seeing. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can
worth checking the driver first.
I'm happy to test one more variant if you want a second data point.
Re: where reranking mattered: worth it? - homelabhenry - 09-29-2026
Good point. The pattern around router policy has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: where reranking mattered: worth it? - tensorfield88 - 09-29-2026
Thanks, that helps. The pattern around consumer GPUs has been pretty similar for me.
Re: where reranking mattered: worth it? - rocmmixer - 09-29-2026
Quote:Thanks, that helps. The pattern around consumer GPUs has been pretty similar for me.
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: where reranking mattered: worth it? - homelabhenry - 09-29-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: where reranking mattered: worth it? - labpath - 09-29-2026
Good point. I ended up in about the same place using llama.cpp.
Re: where reranking mattered: worth it? - homelabhenry - 09-29-2026
That matches what I was seeing. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: where reranking mattered: worth it? - sanepeak48 - 10-01-2026
Did a small comparison this afternoon. On 4090 with Ollama, I saw basically the same behavior. I expected the model choice to matter more than it did.