reranking still worth it for small sets with TensorRT-LLM

[cachebench]

Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once tail latency becomes important.

Interested in tradeoffs more than fan loyalty.