best mini PC for local use: early impressions

[phasespark]

Appreciate the detail. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[byteyard]

Interesting, I hadn't tried that. I ended up in about the same place using SGLang.

[homelabhenry]

Maybe, but I had the opposite result. I get why that seems right, but on Threadripper box the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[homelabhenry]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on A6000, or squeeze more out of the stack and deal with the complexity.

What would you pick if you had to live with it for a few months?

This is mostly for local work rather than public serving.

[dockkite69]

What exact setup are you running? Is this local-only or are you routing anything externally?

[cobalttinker]

A couple things I'd check first. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.