how much orchestration is too much

[southden]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain pat

Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?

[mintstream9]

I hit almost the same issue last night.

[phasestream53]

Need a little more detail here. What model, what runtime, and roughly what context size?

[cudacrate]

Need a little more detail here. What model, what runtime, and roughly what context size?

[lablab]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.

[mlxgarden]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. At this point the real bottleneck is my patience.