+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: MCP / APIs / Tool Calling (https://www.datasculpting.ai/inference/forumdisplay.php?fid=46)
+--- Thread: model merge got better or just different (/showthread.php?tid=243)
Pages:
1
2
Re: model merge got better or just different - thetatinker77 - 09-15-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.
Re: model merge got better or just different - bytestack - 09-15-2026
That matches what I was seeing. I ended up in about the same place using MLX.
Re: model merge got better or just different - mangohouse - 09-15-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at MoE serving. Once you know the plain path works, then add the faster path back in piece by piece.
Re: model merge got better or just different - rocmstack25 - 09-15-2026
Quote:I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at MoE serving. Once you kn
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.
Re: model merge got better or just different - cacheline - 09-15-2026
What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.