tools worth paying for on GB200

[mellowstream]

Thought I'd post a small recipe because a lot of threads ask similar things.

Just sharing a setup that ended up working reasonably well: Llama, LM Studio, M3 Max, with a lot of attention on long context.

Maybe useful as a starting point for newer folks.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

[vramarch]

A couple questions before I guess too hard. Are you testing this on A6000 or something else?

[dockriver]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?

[papermixer4]

Appreciate the detail. I ended up in about the same place using Ollama.

[deepseeked]

What worked for me was pretty simple. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[orbitline]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?

I'd approach it in a boring order. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[nanobrook]

subscribed, curious where this lands.

[modbench]

subscribed, curious where this lands.

[deepseeked]

What exact setup are you running? What model, what runtime, and roughly what context size?

[macrolab]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?