ROCm version that finally behaves: early impressions

[cozywire]

Did a bit of testing and thought I'd post the result here.

Short version: llama.cpp behaved well for me, but only after I stopped ignoring batch size.

Happy to post more details if anyone wants them.

[batchyard]

Did a bit of testing and thought I'd post the result here.

Not sure I agree there. I've seen people blame HBM constraints when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.