+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: what made local AI click for you (/showthread.php?tid=835)
Pages:
1
2
Re: what made local AI click for you - thetagrid15 - 10-11-2026
Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: what made local AI click for you - nanopatch - 10-11-2026
Did a small comparison this afternoon. On 7900 XTX with MLX, I saw basically the same behavior.
Re: what made local AI click for you - quiettoken - 10-11-2026
Good point. The pattern around INT4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: what made local AI click for you - tensorkite - 10-14-2026
following this because I'm in the same boat.
Re: what made local AI click for you - thetagrid15 - 10-14-2026
that fixed it for me too.
Re: what made local AI click for you - batchshed - 10-15-2026
Quote:that fixed it for me too.
Appreciate the detail. The pattern around consumer GPUs has been pretty similar for me.