do small edge boxes make sense for Kimi

[byteyard]

This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like speculative decoding decides everything downstream.

Maybe obvious to some of you, but it clicked for me this week.

[mellowwire98]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.

[mintforge]

Thanks, that helps. I ended up in about the same place using Ollama.

[turboscope5]

that fixed it for me too.

[thetaarch43]

Thanks, that helps. That matches what I saw on A6000.

[amberrunner]

Thanks, that helps. The pattern around tool calling has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.