+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: where reranking mattered on mini PC (/showthread.php?tid=446)
Pages:
1
2
Re: where reranking mattered on mini PC - bytelab - 09-26-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: where reranking mattered on mini PC - tinycontext - 09-26-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: where reranking mattered on mini PC - homelabhenry - 09-27-2026
that fixed it for me too.
Re: where reranking mattered on mini PC - ggufvault - 09-27-2026
Tried to reproduce it just now. On GB200 with llama.cpp, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: where reranking mattered on mini PC - labpath - 09-29-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher.
Re: where reranking mattered on mini PC - embershed99 - 09-29-2026
I don't think that's the whole story. I've seen people blame prefill/decode split when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup. Turns out 'just one more test' was a lie.