+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: reranking still worth it for small sets: setup advice (/showthread.php?tid=883)
reranking still worth it for small sets: setup advice - mellowstream - 10-07-2026
I figured I'd ask here before changing everything.
My current setup is llama.cpp with A6000, and I keep running into tool calling.
If you've tested something similar, what did you keep and what did you stop bothering with?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: reranking still worth it for small sets: setup advice - relaysupply90 - 10-07-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: reranking still worth it for small sets: setup advice - mangopeak72 - 10-08-2026
I'd approach it in a boring order. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: reranking still worth it for small sets: setup advice - edgekite - 10-11-2026
That matches what I was seeing. The pattern around long context has been pretty similar for me.
Re: reranking still worth it for small sets: setup advice - benchplease - 10-11-2026
Good point. I don't think your conclusion is far off.
Re: reranking still worth it for small sets: setup advice - northquill - 10-11-2026
Need a little more detail here. What model, what runtime, and roughly what context size?
Re: reranking still worth it for small sets: setup advice - localcraft17 - 10-14-2026
Appreciate the detail. I ended up in about the same place using llama.cpp.
Re: reranking still worth it for small sets: setup advice - quantghost - 10-14-2026
Quote:Appreciate the detail. I ended up in about the same place using llama.cpp.
Not sure I agree there. I get why that seems right, but on 5090 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: reranking still worth it for small sets: setup advice - thetapath - 10-14-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there.