distill or just quantize for DeepSeek

[quietstack]

Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[cobaltlab]

Tried to reproduce it just now. On M3 Max with vLLM, I saw basically the same behavior.

[torchtrack24]

Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.