+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: reranking still worth it for small sets (/showthread.php?tid=26)
Pages:
1
2
Re: reranking still worth it for small sets - minthall - 08-28-2026
Quote:A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. Once yo
Interesting, I hadn't tried that. I don't think your conclusion is far off.
Re: reranking still worth it for small sets - torchkite - 08-29-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: reranking still worth it for small sets - quantshed - 08-30-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: reranking still worth it for small sets - oxidegrove40 - 08-30-2026
My first guess would be this: If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: reranking still worth it for small sets - bytesupply76 - 08-31-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.