prefill bottlenecks on longer prompts on A6000

[benchplease]

I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the

Appreciate the detail. I ended up in about the same place using vLLM.