08-31-2026, 07:45 PM
Has anyone else run into this?
I'm mostly working with Kimi in llama.cpp on A6000. The part I'm trying to understand is INT4.
Would appreciate practical advice rather than theory.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
I'm mostly working with Kimi in llama.cpp on A6000. The part I'm trying to understand is INT4.
Would appreciate practical advice rather than theory.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.