latency vs throughput for small deployments with TensorRT-LLM

[turboscope5]

Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.