Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
quant choices for 70B local
#6
What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece. At this point the real bottleneck is my patience.
2x3090 • llama.cpp • too much VRAM is still not enough
Reply


Messages In This Thread
quant choices for 70B local - by vrambandit - 09-16-2026, 11:51 PM
Re: quant choices for 70B local - by southtinker - 09-17-2026, 08:34 AM
Re: quant choices for 70B local - by nanobrook - 09-17-2026, 08:57 AM
Re: quant choices for 70B local - by fusedtinker - 09-17-2026, 05:32 PM
Re: quant choices for 70B local - by marbletrack - 09-18-2026, 05:34 PM
Re: quant choices for 70B local - by vrambandit - 09-18-2026, 06:44 PM
Re: quant choices for 70B local - by vrambandit - 09-23-2026, 08:00 AM
Re: quant choices for 70B local - by relaynest82 - 09-20-2026, 06:58 AM
Re: quant choices for 70B local - by southpatch - 09-24-2026, 10:33 AM
Re: quant choices for 70B local - by cudaline - 09-20-2026, 07:24 AM
Re: quant choices for 70B local - by paperrunner - 09-24-2026, 08:46 AM
Re: quant choices for 70B local - by torchshed - 09-25-2026, 11:13 PM
Re: quant choices for 70B local - by llmarch83 - 09-26-2026, 11:44 AM
Re: quant choices for 70B local - by nanobrook - 09-26-2026, 11:48 PM
Re: quant choices for 70B local - by torchshed - 09-27-2026, 02:28 AM
Re: quant choices for 70B local - by fastscope - 09-27-2026, 10:53 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)