prefill bottlenecks on longer prompts: anyone testing this?

[sanescope]

Spent some time on this today and wanted to compare notes.

I've gone back and forth between llama.cpp and TensorRT-LLM for DeepSeek. The interesting difference for me has been batch size.

Interested in tradeoffs more than fan loyalty.

I care more about stability than squeezing the last possible tokens/sec out of it.

[kernelmike]

A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.