what music do you benchmark to on GB200

[tinycontext]

Thought I'd post a small recipe because a lot of threads ask similar things.

This isn't revolutionary, but the combination of PyTorch and 5090 felt surprisingly usable once I tuned around workstation thermals.

If someone wants the exact settings I can clean them up and post them.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

[tinycontext]

Thanks, that helps. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[amberplane]

A couple questions before I guess too hard. Are you testing this on mini PC or something else?

[batchsize]

That matches what I was seeing. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[routecraft]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at structured output. Not glamorous advice, but repeatable baselines save a lot of time.

[rasterhall]

Interesting, I hadn't tried that. That matches what I saw on A6000. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[mintstation99]

Interesting, I hadn't tried that. That matches what I saw on A6000. The biggest change for me was simplifying the stack instead of adding one more opt

That matches what I was seeing. I ended up in about the same place using TensorRT-LLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[promptspark58]

Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[vrambandit]

that fixed it for me too. The logs were somehow calmer than I was.

[stackarch]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.