+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: AMD / ROCm (https://www.datasculpting.ai/inference/forumdisplay.php?fid=32)
+--- Thread: 7900 XTX local stack (/showthread.php?tid=386)
Pages:
1
2
7900 XTX local stack - mintquill - 09-21-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: MLX on B200, keeping token cost in mind from the start.
Maybe useful as a starting point for newer folks.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: 7900 XTX local stack - relaynest82 - 09-21-2026
Quote:Thought I'd post a small recipe because a lot of threads ask similar things.
That matches what I was seeing. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: 7900 XTX local stack - northhall88 - 09-21-2026
Appreciate the detail. I don't think your conclusion is far off.
Re: 7900 XTX local stack - dockworks23 - 09-21-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: 7900 XTX local stack - mintquill - 09-21-2026
What worked for me was pretty simple. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. I keep promising myself this is the last rebuild.
Re: 7900 XTX local stack - tensorworks - 09-21-2026
I think the bottleneck is somewhere else. I've seen people blame HBM constraints when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: 7900 XTX local stack - plaingarden18 - 09-22-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks.
Re: 7900 XTX local stack - rocmriley - 09-25-2026
Quote:What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks.
My first guess would be this: If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: 7900 XTX local stack - tensorworks - 09-25-2026
Good point. The pattern around quant drift has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: 7900 XTX local stack - rocmriley - 09-25-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.