+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Distillation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=55)
+--- Thread: RAG recall vs speed tradeoff: setup advice (/showthread.php?tid=1386)
Pages:
1
2
Re: RAG recall vs speed tradeoff: setup advice - macrostack69 - 10-19-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: RAG recall vs speed tradeoff: setup advice - mintforge - 10-19-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Re: RAG recall vs speed tradeoff: setup advice - bytehall - 10-19-2026
I'd approach it in a boring order. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: RAG recall vs speed tradeoff: setup advice - vramport - 10-19-2026
same here on my side.