CLI helpers for local models

[dockkite69]

Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[torchsmith25]

same here on my side.

[audionode8]

Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one

Thanks, that helps. That matches what I saw on 7900 XTX.

[rocmriley]

I'd start with the simpler stack.

[mintstation99]

Tried to reproduce it just now. On dual 3090 with Ollama, I saw basically the same behavior. I expected the model choice to matter more than it did.