+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: CUDA update broke my throughput with TensorRT-LLM (/showthread.php?tid=43)
Pages:
1
2
Re: CUDA update broke my throughput with TensorRT-LLM - latentgarden - 09-03-2026
Thanks, that helps. I ended up in about the same place using llama.cpp. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: CUDA update broke my throughput with TensorRT-LLM - phaseport36 - 09-03-2026
Did a small comparison this afternoon. On A6000 with llama.cpp, I saw basically the same behavior. The main thing that moved was latency, not quality.
I'm happy to test one more variant if you want a second data point.