+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: reranking still worth it for small sets: early impressions (/showthread.php?tid=219)
Pages:
1
2
reranking still worth it for small sets: early impressions - phasespark - 09-10-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Kimi using vLLM on 7900 XTX. The headline result was better than I expected, but structured output still ended up being the limiting factor.
Happy to post more details if anyone wants them.
Re: reranking still worth it for small sets: early impressions - cozysmith - 09-11-2026
Appreciate the detail. I don't think your conclusion is far off.
Re: reranking still worth it for small sets: early impressions - thetaember38 - 09-11-2026
A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: reranking still worth it for small sets: early impressions - batchsize - 09-12-2026
That matches what I was seeing. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: reranking still worth it for small sets: early impressions - sanepeak48 - 09-13-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: reranking still worth it for small sets: early impressions - deepseeked - 09-16-2026
Quote:A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: reranking still worth it for small sets: early impressions - latentcraft - 09-19-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: reranking still worth it for small sets: early impressions - turbogrid11 - 09-21-2026
Appreciate the detail. I don't think your conclusion is far off.
Re: reranking still worth it for small sets: early impressions - prompttinker44 - 09-21-2026
Appreciate the detail. That matches what I saw on GB200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: reranking still worth it for small sets: early impressions - fusedriver - 09-22-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?