paged attention observations for Mistral

[cacheline]

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.

[cacheline]

I'd approach it in a boring order. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not

What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?

[edgekite]

Need a little more detail here. Are you testing this on 5090 or something else? At this point the real bottleneck is my patience.