Jetson still a thing with MLX

[cobalttinker]

Thought I'd post a small recipe because a lot of threads ask similar things.

This isn't revolutionary, but the combination of MLX and M3 Max felt surprisingly usable once I tuned around KV cache pressure.

If someone wants the exact settings I can clean them up and post them.

This is mostly for local work rather than public serving.

[cacheplane98]

Thought I'd post a small recipe because a lot of threads ask similar things.

Need a little more detail here. Is this local-only or are you routing anything externally?

[mintquill]

Thought I'd post a small recipe because a lot of threads ask similar things.

following this because I'm in the same boat.

[promptfield]

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.

[homelabhenry]

A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[cobalttinker]

subscribed, curious where this lands.