+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: Blackwell deployment chatter (/showthread.php?tid=578)
Pages:
1
2
Blackwell deployment chatter - phaseport36 - 09-30-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: MLX on 5090, keeping long context in mind from the start.
Maybe useful as a starting point for newer folks.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: Blackwell deployment chatter - amberpatch25 - 09-30-2026
Thanks, that helps. The pattern around workstation thermals has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Blackwell deployment chatter - kernelmike - 10-01-2026
Quote:Thanks, that helps. The pattern around workstation thermals has been pretty similar for me. The biggest change for me was simplifying the stack instea
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at MoE serving. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Blackwell deployment chatter - mintforge - 10-01-2026
Thanks, that helps. The pattern around long context has been pretty similar for me.
Re: Blackwell deployment chatter - kernelmike - 10-01-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Blackwell deployment chatter - macronest - 10-02-2026
What worked for me was pretty simple. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Blackwell deployment chatter - batchpeak - 10-02-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: Blackwell deployment chatter - dockember - 10-02-2026
following this because I'm in the same boat.
Re: Blackwell deployment chatter - thetashed24 - 10-02-2026
Thanks, that helps. That matches what I saw on GB200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Blackwell deployment chatter - audiopath66 - 10-02-2026
I hit almost the same issue last night.