+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Agents (https://www.datasculpting.ai/inference/forumdisplay.php?fid=44)
+--- Thread: distill or just quantize for DeepSeek (/showthread.php?tid=37)
Pages:
1
2
Re: distill or just quantize for DeepSeek - quietstack - 08-30-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: distill or just quantize for DeepSeek - cobaltlab - 08-30-2026
Tried to reproduce it just now. On M3 Max with vLLM, I saw basically the same behavior.
Re: distill or just quantize for DeepSeek - torchtrack24 - 08-30-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.