+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Search / Retrieval / Reranking discussion on 4090 (/showthread.php?tid=769)
Search / Retrieval / Reranking discussion on 4090 - vramscope - 10-05-2026
Posting this in case it saves someone else a little time.
If you were starting from scratch today for this use case, what would you choose first and what would you ignore?
I'm especially interested in mistakes worth avoiding.
Re: Search / Retrieval / Reranking discussion on 4090 - modrunner27 - 10-05-2026
Quote:Posting this in case it saves someone else a little time.
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Search / Retrieval / Reranking discussion on 4090 - promptfield - 10-08-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: Search / Retrieval / Reranking discussion on 4090 - tinycontext - 10-11-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at reranking. Once you know the plain path works, then add the faster path back in piece by piece.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: Search / Retrieval / Reranking discussion on 4090 - mellowsupply - 10-15-2026
Appreciate the detail. That matches what I saw on B200. I keep promising myself this is the last rebuild.
Re: Search / Retrieval / Reranking discussion on 4090 - homelabhenry - 10-15-2026
Thanks, that helps. The pattern around VRAM headroom has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.