reranking still worth it for small sets

[circuitkite]

What exact setup are you running? Are you testing this on 7900 XTX or something else?

[llmarch83]

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[thetanest]

Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.

[southspark]

That matches what I was seeing. The pattern around MoE serving has been pretty similar for me.

[rocmstack25]

Thanks, that helps. I ended up in about the same place using SGLang.

[latentgarden]

Thanks, that helps. I ended up in about the same place using SGLang.

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.