Inference › Training & Customization › Distillation v
RAG recall vs speed tradeoff

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
RAG recall vs speed tradeoff
batchsize Offline
Member
***
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation: 0
#10
09-10-2026, 05:31 AM
Appreciate the detail. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
throughput matters, but tail latency matters too
Find
Reply


Messages In This Thread
RAG recall vs speed tradeoff - by cobaltworks25 - 09-05-2026, 05:40 PM
Re: RAG recall vs speed tradeoff - by llmarch51 - 09-08-2026, 06:47 AM
Re: RAG recall vs speed tradeoff - by batchsize - 09-10-2026, 05:31 AM
Re: RAG recall vs speed tradeoff - by localpatch - 09-10-2026, 06:02 PM
Re: RAG recall vs speed tradeoff - by bytepilot - 09-09-2026, 03:06 PM
Re: RAG recall vs speed tradeoff - by turboscope - 09-10-2026, 03:49 AM
Re: RAG recall vs speed tradeoff - by rocmmixer - 09-10-2026, 04:30 AM
Re: RAG recall vs speed tradeoff - by vectorpoint - 09-11-2026, 06:59 PM
Re: RAG recall vs speed tradeoff - by fastbench - 09-10-2026, 06:30 PM
Re: RAG recall vs speed tradeoff - by plainworks - 09-11-2026, 07:42 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)