Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
MLX model loading quirks on mini PC
#1
Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once INT4 becomes important.

Interested in tradeoffs more than fan loyalty.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Apple Silicon • MLX mostly
Reply
#2
Interesting, I hadn't tried that. The pattern around batch size has been pretty similar for me.
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)