+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Recipes & Configurations (https://www.datasculpting.ai/inference/forumdisplay.php?fid=52)
+---- Thread: model fallback rules that work (/showthread.php?tid=968)
Pages:
1
2
Re: model fallback rules that work - homelabhenry - 10-18-2026
Good point. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: model fallback rules that work - phaseport36 - 10-18-2026
following this because I'm in the same boat.
Re: model fallback rules that work - turboscope5 - 10-18-2026
Not sure I agree there. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: model fallback rules that work - sparrowshed - 10-18-2026
Quote:Not sure I agree there. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't gener
I tested this tonight. On dual 3090 with TensorRT-LLM, I saw basically the same behavior.
Re: model fallback rules that work - macropilot - 10-19-2026
Interesting, I hadn't tried that. That matches what I saw on dual 3090.
Re: model fallback rules that work - amberrunner - 10-19-2026
Quote:[quote]Not sure I agree there. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Could be a workload difference, but I wouldn'
Can you fill in a bit more context? Are you testing this on A6000 or something else?
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.