+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: PCIe lane weirdness with inference loads (/showthread.php?tid=482)
Pages:
1
2
PCIe lane weirdness with inference loads - dockkite69 - 09-25-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on A6000, or squeeze more out of the stack and deal with the complexity.
Trying to avoid rebuilding the whole thing twice.
Re: PCIe lane weirdness with inference loads - torchshed - 09-25-2026
What exact setup are you running? Are you testing this on mini PC or something else?
Re: PCIe lane weirdness with inference loads - rocmriley - 09-26-2026
Maybe, but I had the opposite result. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: PCIe lane weirdness with inference loads - rocmpath - 09-26-2026
Quote:Maybe, but I had the opposite result. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Could be a workload difference, but
Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: PCIe lane weirdness with inference loads - turboyard94 - 09-26-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: PCIe lane weirdness with inference loads - embersmith - 09-26-2026
I'd approach it in a boring order. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: PCIe lane weirdness with inference loads - signaltrack44 - 09-26-2026
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: PCIe lane weirdness with inference loads - rocmriley - 09-27-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. Once you know the plain path works, then add the faster path back in piece by piece.
Re: PCIe lane weirdness with inference loads - thetarunner - 09-27-2026
Good point. That matches what I saw on 7900 XTX. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: PCIe lane weirdness with inference loads - dockkite69 - 09-27-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.