prefill bottlenecks on longer prompts on A6000

[mintstream9]

This keeps coming up in conversation so I figured I'd start a thread.

Good point. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.