best mini PC for local use: early impressions

[vramghost]

Interesting, I hadn't tried that. I ended up in about the same place using llama.cpp. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.