+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Agents (https://www.datasculpting.ai/inference/forumdisplay.php?fid=44)
+--- Thread: distill or just quantize for Mistral (/showthread.php?tid=1144)
Pages:
1
2
Re: distill or just quantize for Mistral - amberpatch25 - 10-19-2026
Tried to reproduce it just now. On A6000 with Ollama, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed. The logs were somehow calmer than I was.
Re: distill or just quantize for Mistral - phasescope - 10-19-2026
subscribed, curious where this lands.