how much orchestration is too much: what are you seeing?

[cudacrate]

I don't think that's the whole story. I get why that seems right, but on GB200 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[dockworks23]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.

[ggufvault]

same here on my side.

[phasestream53]

Maybe, but I had the opposite result. I've seen people blame reranking when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet. Turns out 'just one more test' was a lie.

[llm_wren]

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.