|
quant choices for 70B local on GB200
|
10-13-2026, 05:31 PM
I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
2x3090 • llama.cpp • too much VRAM is still not enough
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
quant choices for 70B local on GB200 - by cudaline - 10-13-2026, 04:23 AM
Re: quant choices for 70B local on GB200 - by cobalttinker - 10-13-2026, 04:51 AM
Re: quant choices for 70B local on GB200 - by vrambandit - 10-13-2026, 05:31 PM
Re: quant choices for 70B local on GB200 - by paperrunner - 10-15-2026, 06:16 AM
Re: quant choices for 70B local on GB200 - by fusedtinker - 10-15-2026, 06:59 AM
Re: quant choices for 70B local on GB200 - by benchplease - 10-15-2026, 08:27 AM
|
