tokens/sec tuning ideas: what are you seeing?

[turbosupply83]

My first guess would be this: If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.