|
paged attention observations
|
10-18-2026, 11:01 AM
Quote:My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of the I don't think that's the whole story. I've seen people blame consumer GPUs when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
measure twice, benchmark three times
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
paged attention observations - by cacheline - 10-16-2026, 04:39 AM
Re: paged attention observations - by cacheline - 10-16-2026, 04:57 AM
Re: paged attention observations - by cacheline - 10-17-2026, 05:46 AM
Re: paged attention observations - by cacheline - 10-17-2026, 06:34 AM
Re: paged attention observations - by cudascope - 10-17-2026, 07:21 AM
Re: paged attention observations - by cacheline - 10-17-2026, 07:43 AM
Re: paged attention observations - by northquill - 10-17-2026, 10:27 AM
Re: paged attention observations - by cacheline - 10-18-2026, 11:01 AM
Re: paged attention observations - by vectorhouse71 - 10-18-2026, 11:18 PM
Re: paged attention observations - by cacheline - 10-19-2026, 03:42 AM
Re: paged attention observations - by audioriver - 10-19-2026, 04:23 AM
|
