+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: paged attention observations (/showthread.php?tid=1399)
paged attention observations - cacheline - 10-18-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate KV cache pressure until they actually try to run this stuff for more than one afternoon.
Wondering if this is just where the whole space is heading now.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: paged attention observations - edgecoast - 10-18-2026
Quote:That matches what I was seeing. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can
My first guess would be this: On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: paged attention observations - dockkite69 - 10-18-2026
Thanks, that helps. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: paged attention observations - cacheline - 10-18-2026
Quote:That matches what I was seeing. The pattern around quant drift has been pretty similar for me. I got better results once I stopped chasing one metric
following this because I'm in the same boat.
Re: paged attention observations - sanepeak48 - 10-18-2026
Thanks, that helps. I ended up in about the same place using MLX. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: paged attention observations - cacheline - 10-19-2026
Did a small comparison this afternoon. On A6000 with TensorRT-LLM, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: paged attention observations - cacheline - 10-19-2026
Quote:[quote]That matches what I was seeing. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumpt
I hit almost the same issue last night.
Re: paged attention observations - quantnest - 10-19-2026
That matches what I was seeing. The pattern around quant drift has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: paged attention observations - cacheline - 10-19-2026
That matches what I was seeing. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.