Ollama for power users

[dockridge]

Thought I'd post a small recipe because a lot of threads ask similar things.

This isn't revolutionary, but the combination of MLX and 7900 XTX felt surprisingly usable once I tuned around long context.

Feel free to tear it apart if you see something dumb.

[vectorhouse71]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.

[oxidemixer]

A couple things I'd check first. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.

[fusedhall]

A couple questions before I guess too hard. Are you testing this on A6000 or something else?

[edgecoast]

What worked for me was pretty simple. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[ggufwire22]

that fixed it for me too.

[dockridge]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.