+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: reranking still worth it for small sets (/showthread.php?tid=205)
reranking still worth it for small sets - homelabhenry - 09-09-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on mini PC, or squeeze more out of the stack and deal with the complexity.
Trying to avoid rebuilding the whole thing twice.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: reranking still worth it for small sets - flintriver50 - 09-09-2026
What worked for me was pretty simple. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: reranking still worth it for small sets - rocmmixer - 09-09-2026
Ran a quick check on my box. On B200 with vLLM, I saw basically the same behavior. I expected the model choice to matter more than it did.