+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: paged attention observations for Mistral (/showthread.php?tid=1320)
Pages:
1
2
Re: paged attention observations for Mistral - cacheline - 10-19-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: paged attention observations for Mistral - cacheline - 10-19-2026
Quote:I'd approach it in a boring order. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: paged attention observations for Mistral - edgekite - 10-19-2026
Need a little more detail here. Are you testing this on 5090 or something else? At this point the real bottleneck is my patience.