Inference › Optimization & Performance › Context / KV Cache v
paged attention observations

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
paged attention observations
cacheline Offline
Senior Member
****
Posts: 341
Threads: 51
Joined: Sep 2026
Reputation: 0
#2
10-16-2026, 04:57 AM
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
measure twice, benchmark three times
Find
Reply


Messages In This Thread
paged attention observations - by cacheline - 10-16-2026, 04:39 AM
Re: paged attention observations - by cacheline - 10-16-2026, 04:57 AM
Re: paged attention observations - by cacheline - 10-17-2026, 05:46 AM
Re: paged attention observations - by cacheline - 10-17-2026, 06:34 AM
Re: paged attention observations - by cudascope - 10-17-2026, 07:21 AM
Re: paged attention observations - by cacheline - 10-17-2026, 07:43 AM
Re: paged attention observations - by northquill - 10-17-2026, 10:27 AM
Re: paged attention observations - by cacheline - 10-18-2026, 11:01 AM
Re: paged attention observations - by cacheline - 10-19-2026, 03:42 AM
Re: paged attention observations - by audioriver - 10-19-2026, 04:23 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)