ROCm version that finally behaves: anyone testing this?

[cozykite]

A couple things I'd check first. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

I'm happy to test one more variant if you want a second data point.

[mangofield33]

Spent some time on this today and wanted to compare notes.

I've gone back and forth between MLX and MLX for DeepSeek. The interesting difference for me has been reranking.

Interested in tradeoffs more than fan loyalty.

This is mostly for local work rather than public serving.

[rocmmixer]

Need a little more detail here. Is this local-only or are you routing anything externally?