10-06-2026, 09:42 AM
Spent some time on this today and wanted to compare notes.
I've gone back and forth between MLX and llama.cpp for Kimi. The interesting difference for me has been prefill/decode split.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
I've gone back and forth between MLX and llama.cpp for Kimi. The interesting difference for me has been prefill/decode split.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Mostly CUDA, sometimes wrong, usually measured