model fallback rules that work: setup advice

[mellowstream]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.

Good point. That matches what I saw on mini PC. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[cobaltworks25]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.

What exact setup are you running? What model, what runtime, and roughly what context size?