NVIDIA driver regression with SGLang

[dockember]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around PyTorch and B200.

I'm especially interested in mistakes worth avoiding.

[dockember]

Appreciate the detail. The pattern around embeddings has been pretty similar for me.

[kernelmike]

Appreciate the detail. The pattern around embeddings has been pretty similar for me.

that fixed it for me too.

[oxidemixer]

That matches what I was seeing. The pattern around prefix caching has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[mintforge]

that fixed it for me too.

[kernelmike]

that fixed it for me too.

Good point. The pattern around long context has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[sparrowkite53]

I hit almost the same issue last night.

[thetapath]

worth checking the driver first.

[pixelspark]

I hit almost the same issue last night.

[thetapath]

My first guess would be this: If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious.