Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
paged attention observations with MLX
#1
This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like batch size decides everything downstream.

Curious whether that lines up with what other people are seeing.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
measure twice, benchmark three times
Reply


Messages In This Thread
paged attention observations with MLX - by cacheline - 10-05-2026, 05:54 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)