Inference › Optimization & Performance › Quantization v
quant choices for 70B local

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
quant choices for 70B local
vrambandit Offline
Member
***
Posts: 168
Threads: 14
Joined: Sep 2026
Reputation: 0
#9
09-23-2026, 08:00 AM
following this because I'm in the same boat.
2x3090 • llama.cpp • too much VRAM is still not enough
Find
Reply


Messages In This Thread
quant choices for 70B local - by vrambandit - 09-16-2026, 11:51 PM
Re: quant choices for 70B local - by southtinker - 09-17-2026, 08:34 AM
Re: quant choices for 70B local - by nanobrook - 09-17-2026, 08:57 AM
Re: quant choices for 70B local - by fusedtinker - 09-17-2026, 05:32 PM
Re: quant choices for 70B local - by marbletrack - 09-18-2026, 05:34 PM
Re: quant choices for 70B local - by vrambandit - 09-18-2026, 06:44 PM
Re: quant choices for 70B local - by vrambandit - 09-23-2026, 08:00 AM
Re: quant choices for 70B local - by relaynest82 - 09-20-2026, 06:58 AM
Re: quant choices for 70B local - by southpatch - 09-24-2026, 10:33 AM
Re: quant choices for 70B local - by cudaline - 09-20-2026, 07:24 AM
Re: quant choices for 70B local - by paperrunner - 09-24-2026, 08:46 AM
Re: quant choices for 70B local - by torchshed - 09-25-2026, 11:13 PM
Re: quant choices for 70B local - by llmarch83 - 09-26-2026, 11:44 AM
Re: quant choices for 70B local - by nanobrook - 09-26-2026, 11:48 PM
Re: quant choices for 70B local - by torchshed - 09-27-2026, 02:28 AM
Re: quant choices for 70B local - by fastscope - 09-27-2026, 10:53 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)