Inference › Inference Engines › llama.cpp / GGUF v
prefix caching in llama.cpp with MLX

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
prefix caching in llama.cpp with MLX
thetagrid15 Offline
Member
***
Posts: 68
Threads: 3
Joined: Sep 2026
Reputation: 0
#4
09-23-2026, 04:29 AM
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.
Find
Reply


Messages In This Thread
Re: prefix caching in llama.cpp with MLX - by thetagrid15 - 09-23-2026, 04:29 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)