+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: AMD / ROCm (https://www.datasculpting.ai/inference/forumdisplay.php?fid=32)
+--- Thread: HIP kernel compile issue for Mixtral (/showthread.php?tid=1336)
Pages:
1
2
HIP kernel compile issue for Mixtral - latentgarden - 10-16-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Kimi, TensorRT-LLM, dual 3090, with a lot of attention on continuous batching.
Maybe useful as a starting point for newer folks.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs. I keep promising myself this is the last rebuild.
Re: HIP kernel compile issue for Mixtral - phasescope - 10-18-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at HBM constraints.
Re: HIP kernel compile issue for Mixtral - quantshed - 10-18-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: HIP kernel compile issue for Mixtral - cozykite - 10-18-2026
A couple questions before I guess too hard. Are you testing this on dual 3090 or something else?
Re: HIP kernel compile issue for Mixtral - mangofield33 - 10-19-2026
Quote:My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. A lot of these threads end
Need a little more detail here. Are you testing this on Threadripper box or something else?
Re: HIP kernel compile issue for Mixtral - fastquill74 - 10-19-2026
Quote:A couple questions before I guess too hard. Are you testing this on dual 3090 or something else?
Thanks, that helps. I ended up in about the same place using Ollama. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: HIP kernel compile issue for Mixtral - macropoint - 10-19-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: HIP kernel compile issue for Mixtral - thetaember38 - 10-19-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: HIP kernel compile issue for Mixtral - mangofield33 - 10-19-2026
I'd start with the simpler stack.
I'm happy to test one more variant if you want a second data point.
Re: HIP kernel compile issue for Mixtral - fastquill74 - 10-19-2026
Appreciate the detail. The pattern around INT4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
I'm happy to test one more variant if you want a second data point.