+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Recipes & Configurations (https://www.datasculpting.ai/inference/forumdisplay.php?fid=52)
+---- Thread: simple router policy ideas (/showthread.php?tid=586)
simple router policy ideas - northgarden37 - 09-29-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of Ollama and M3 Max felt surprisingly usable once I tuned around reranking.
If someone wants the exact settings I can clean them up and post them.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: simple router policy ideas - stackpilot - 10-01-2026
Good point. I ended up in about the same place using TensorRT-LLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: simple router policy ideas - deepseeked - 10-01-2026
Thanks, that helps. I ended up in about the same place using llama.cpp.
Re: simple router policy ideas - audioriver - 10-02-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: simple router policy ideas - macrolab - 10-02-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: simple router policy ideas - lablab - 10-02-2026
following this because I'm in the same boat.