+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Distillation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=55)
+--- Thread: RAG recall vs speed tradeoff on Threadripper box (/showthread.php?tid=515)
Pages:
1
2
Re: RAG recall vs speed tradeoff on Threadripper box - llm_wren - 10-02-2026
Quote:Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: RAG recall vs speed tradeoff on Threadripper box - turboarch95 - 10-03-2026
Need a little more detail here. Are you testing this on B200 or something else?