+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: CPUs / NPUs / Accelerators (https://www.datasculpting.ai/inference/forumdisplay.php?fid=30)
+--- Thread: do small edge boxes make sense for Kimi (/showthread.php?tid=589)
do small edge boxes make sense for Kimi - byteyard - 09-29-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like speculative decoding decides everything downstream.
Maybe obvious to some of you, but it clicked for me this week.
Re: do small edge boxes make sense for Kimi - mellowwire98 - 09-30-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.
Re: do small edge boxes make sense for Kimi - mintforge - 09-30-2026
Thanks, that helps. I ended up in about the same place using Ollama.
Re: do small edge boxes make sense for Kimi - turboscope5 - 09-30-2026
that fixed it for me too.
Re: do small edge boxes make sense for Kimi - thetaarch43 - 10-01-2026
Thanks, that helps. That matches what I saw on A6000.
Re: do small edge boxes make sense for Kimi - amberrunner - 10-04-2026
Thanks, that helps. The pattern around tool calling has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.