best mini PC for local use for Kimi

[vramghost]

Posting this in case it saves someone else a little time.

If you were starting from scratch today for this use case, what would you choose first and what would you ignore?

Simple and boring is a valid answer.

I care more about stability than squeezing the last possible tokens/sec out of it.

If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.

[thetatinker77]

Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?

[vramghost]

That matches what I was seeing. I ended up in about the same place using MLX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[thetatinker77]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift.

[mintquill]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.

[phasespark]

Thanks, that helps. I ended up in about the same place using TensorRT-LLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[byteyard]

Appreciate the detail. I ended up in about the same place using SGLang.

[quantsupply]

A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[homelabhenry]

I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet. The logs were somehow calmer than I was.

[cobalttinker]

following this because I'm in the same boat.