+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Quantization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=40)
+--- Thread: quant choices for 70B local with PyTorch (/showthread.php?tid=1426)
Pages:
1
2
Re: quant choices for 70B local with PyTorch - relaynest82 - 10-19-2026
Quote:Quote:following this because I'm in the same boat.
What exact setup are you running? Are you testing this on 7900 XTX or something else?