quiet dual-GPU build advice

[stackpilot]

This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like NVFP4 decides everything downstream.

Maybe obvious to some of you, but it clicked for me this week.

I care more about stability than squeezing the last possible tokens/sec out of it.

[homelabhenry]

This keeps coming up in conversation so I figured I'd start a thread.

worth checking the driver first.

[emberpeak]

This keeps coming up in conversation so I figured I'd start a thread.

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal.

[kernelarch]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.

[sanevault]

Good point. That matches what I saw on 4090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[bytegarden73]

Good point. That matches what I saw on 4090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look co

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4. Not glamorous advice, but repeatable baselines save a lot of time.

[routeforge64]

That matches what I was seeing. I don't think your conclusion is far off.

[batchyard]

worth checking the driver first.

[homelabhenry]

Can you fill in a bit more context? Are you testing this on B200 or something else?

[cudaline]

What worked for me was pretty simple. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives.