+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Distillation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=55)
+--- Thread: RAG recall vs speed tradeoff (/showthread.php?tid=141)
Pages:
1
2
Re: RAG recall vs speed tradeoff - localpatch - 09-10-2026
Quote:Appreciate the detail. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whethe
worth checking the driver first.
Re: RAG recall vs speed tradeoff - fastbench - 09-10-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: RAG recall vs speed tradeoff - vectorpoint - 09-11-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Re: RAG recall vs speed tradeoff - plainworks - 09-11-2026
Not sure I agree there. I get why that seems right, but on GB200 the bottleneck felt elsewhere.