+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Recipes & Configurations (https://www.datasculpting.ai/inference/forumdisplay.php?fid=52)
+---- Thread: simple router policy ideas (/showthread.php?tid=213)
simple router policy ideas - tinycontext - 09-11-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about HBM constraints.
Simple and boring is a valid answer.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Re: simple router policy ideas - cozygrid12 - 09-12-2026
Quote:Posting this in case it saves someone else a little time.
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: simple router policy ideas - dockridge - 09-12-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. I keep promising myself this is the last rebuild.