+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Dataset Creation / Curation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=56)
+--- Thread: reranking still worth it for small sets with TensorRT-LLM (/showthread.php?tid=1338)
reranking still worth it for small sets with TensorRT-LLM - cachebench - 10-16-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once tail latency becomes important.
Interested in tradeoffs more than fan loyalty.