+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: CUDA update broke my throughput (/showthread.php?tid=1381)
Pages:
1
2
Re: CUDA update broke my throughput - cobaltspark - 10-19-2026
Quote:Curious what people here are seeing.
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: CUDA update broke my throughput - mintforge - 10-19-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs.
Re: CUDA update broke my throughput - dockember - 10-19-2026
I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: CUDA update broke my throughput - nanobrook - 10-19-2026
Quote:Interesting, I hadn't tried that. That matches what I saw on M3 Max.
Maybe, but I had the opposite result. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.