model merge got better or just different: anyone testing this?

[relayforge]

Maybe I'm missing something obvious.

My current setup is PyTorch with A6000, and I keep running into prefill/decode split.

Would appreciate practical advice rather than theory.

[phasespark]

Good point. I ended up in about the same place using Ollama. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[vramforge]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.

If you post your exact settings, people here can probably narrow it down pretty fast.