+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: paged attention observations (/showthread.php?tid=413)
Pages:
1
2
Re: paged attention observations - cacheline - 09-29-2026
worth checking the driver first.
Re: paged attention observations - sparrowhall - 10-02-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: paged attention observations - cacheline - 10-02-2026
A couple things I'd check first. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.