serving on Blackwell

[torchtrack24]

I figured I'd ask here before changing everything.

My current setup is Ollama with B200, and I keep running into consumer GPUs.

Would appreciate practical advice rather than theory.

[kernelmike]

I think the bottleneck is somewhere else. I've seen people blame speculative decoding when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[tensorkite]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.

[syshouse]

A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.

[kernelmike]

Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[kernelpeak]

That matches what I was seeing. That matches what I saw on 4090.

[kernelgrid]

Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?