+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Recipes & Configurations (https://www.datasculpting.ai/inference/forumdisplay.php?fid=52)
+---- Thread: simple router policy ideas: what are you seeing? (/showthread.php?tid=270)
Pages:
1
2
Re: simple router policy ideas: what are you seeing? - fastquill74 - 09-17-2026
I think the bottleneck is somewhere else. I've seen people blame MoE serving when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: simple router policy ideas: what are you seeing? - modrunner27 - 09-18-2026
Quote:I think the bottleneck is somewhere else. I've seen people blame MoE serving when the real issue was much more mundane. Worth testing the simpler expl
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: simple router policy ideas: what are you seeing? - audionode8 - 09-18-2026
That matches what I was seeing. That matches what I saw on B200. The logs were somehow calmer than I was.
Re: simple router policy ideas: what are you seeing? - visionforge - 09-18-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.