distillation that keeps useful behavior for Llama

[dockember]

My first guess would be this: If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.