distill or just quantize with Ollama

[torchtrack24]

I'd start with the simpler stack.

[northquill]

I'd start with the simpler stack.

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.

[plainworks]

Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.