+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: NVIDIA driver regression for Phi (/showthread.php?tid=1287)
Pages:
1
2
NVIDIA driver regression for Phi - edgecoast - 10-16-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Gemma using vLLM on M3 Max. The headline result was better than I expected, but reranking still ended up being the limiting factor.
Curious if anyone else landed in the same place.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: NVIDIA driver regression for Phi - torchshed - 10-16-2026
My first guess would be this: If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: NVIDIA driver regression for Phi - kernelmike - 10-18-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: NVIDIA driver regression for Phi - amberpatch25 - 10-18-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: NVIDIA driver regression for Phi - cozysmith - 10-18-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal.
Re: NVIDIA driver regression for Phi - edgecoast - 10-19-2026
Quote:What exact setup are you running? Are you testing this on 4090 or something else?
Good point. The pattern around tail latency has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: NVIDIA driver regression for Phi - thetapath - 10-19-2026
Interesting, I hadn't tried that. That matches what I saw on dual 3090.
Re: NVIDIA driver regression for Phi - mintforge - 10-19-2026
I don't think that's the whole story. I've seen people blame quant drift when the real issue was much more mundane.
Re: NVIDIA driver regression for Phi - cobaltspark - 10-19-2026
What exact setup are you running? Are you testing this on 4090 or something else?
Re: NVIDIA driver regression for Phi - vectorpoint - 10-19-2026
I tested this tonight. On B200 with MLX, I saw basically the same behavior.