+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: serving on Blackwell (/showthread.php?tid=948)
serving on Blackwell - torchtrack24 - 10-09-2026
I figured I'd ask here before changing everything.
My current setup is Ollama with B200, and I keep running into consumer GPUs.
Would appreciate practical advice rather than theory.
Re: serving on Blackwell - kernelmike - 10-11-2026
I think the bottleneck is somewhere else. I've seen people blame speculative decoding when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: serving on Blackwell - tensorkite - 10-11-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.
Re: serving on Blackwell - syshouse - 10-13-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: serving on Blackwell - kernelmike - 10-14-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: serving on Blackwell - kernelpeak - 10-14-2026
That matches what I was seeing. That matches what I saw on 4090.
Re: serving on Blackwell - kernelgrid - 10-14-2026
Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?