+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: serving on Blackwell for Phi (/showthread.php?tid=1285)
Pages:
1
2
serving on Blackwell for Phi - pixelcrate82 - 10-16-2026
I figured I'd ask here before changing everything.
I'm mostly working with Phi in TensorRT-LLM on Threadripper box. The part I'm trying to understand is quant drift.
Would appreciate practical advice rather than theory.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: serving on Blackwell for Phi - macrotinker - 10-16-2026
What worked for me was pretty simple. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: serving on Blackwell for Phi - sysstation69 - 10-16-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: serving on Blackwell for Phi - northcraft90 - 10-16-2026
Need a little more detail here. Is this local-only or are you routing anything externally?
Re: serving on Blackwell for Phi - kernelpeak - 10-18-2026
Quote:Need a little more detail here. Is this local-only or are you routing anything externally?
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: serving on Blackwell for Phi - torchtrack24 - 10-18-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher.
Re: serving on Blackwell for Phi - kernelgrid - 10-18-2026
Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: serving on Blackwell for Phi - torchtrack24 - 10-18-2026
What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: serving on Blackwell for Phi - routecrate - 10-18-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal.
Re: serving on Blackwell for Phi - rocmmixer - 10-18-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.