quant choices for 70B local: what are you seeing?

[vectorport]

Spent some time on this today and wanted to compare notes.

I've gone back and forth between PyTorch and vLLM for DeepSeek. The interesting difference for me has been VRAM headroom.

Interested in tradeoffs more than fan loyalty.

I care more about stability than squeezing the last possible tokens/sec out of it.