+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Agents (https://www.datasculpting.ai/inference/forumdisplay.php?fid=44)
+--- Thread: distill or just quantize with Ollama (/showthread.php?tid=859)
Pages:
1
2
Re: distill or just quantize with Ollama - torchtrack24 - 10-16-2026
I'd start with the simpler stack.
Re: distill or just quantize with Ollama - northquill - 10-16-2026
Quote:I'd start with the simpler stack.
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: distill or just quantize with Ollama - plainworks - 10-16-2026
Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.