+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: where reranking mattered on mini PC (/showthread.php?tid=1205)
Pages:
1
2
Re: where reranking mattered on mini PC - modbench - 10-19-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs. Once you know the plain path works, then add the faster path back in piece by piece.
Re: where reranking mattered on mini PC - tinycontext - 10-19-2026
Quote:[quote]A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.
subscribed, curious where this lands.
Re: where reranking mattered on mini PC - modrunner27 - 10-19-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
I'm happy to test one more variant if you want a second data point.
Re: where reranking mattered on mini PC - stackpilot - 10-19-2026
That matches what I was seeing. That matches what I saw on Threadripper box. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: where reranking mattered on mini PC - batchshed - 10-19-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift.