Inference › Optimization & Performance › Quantization v
quant choices for 70B local

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
quant choices for 70B local
turboscope Offline
Junior Member
**
Posts: 35
Threads: 8
Joined: Aug 2026
Reputation: 0
#12
10-18-2026, 08:08 PM
Interesting, I hadn't tried that. The pattern around tokens/sec has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing. The logs were somehow calmer than I was.
Find
Reply


Messages In This Thread
quant choices for 70B local - by southden - 10-11-2026, 01:50 AM
Re: quant choices for 70B local - by vramscope - 10-11-2026, 01:50 PM
Re: quant choices for 70B local - by benchplease - 10-12-2026, 02:24 AM
Re: quant choices for 70B local - by northquill - 10-12-2026, 10:42 AM
Re: quant choices for 70B local - by southtinker - 10-12-2026, 10:59 AM
Re: quant choices for 70B local - by torchshed - 10-15-2026, 11:45 AM
Re: quant choices for 70B local - by fusedtinker - 10-18-2026, 12:08 PM
Re: quant choices for 70B local - by benchplease - 10-18-2026, 01:21 PM
Re: quant choices for 70B local - by vrambandit - 10-18-2026, 04:51 PM
Re: quant choices for 70B local - by turboscope - 10-18-2026, 08:08 PM
Re: quant choices for 70B local - by cacheworks - 10-18-2026, 09:02 PM
Re: quant choices for 70B local - by relaynest82 - 10-18-2026, 09:24 PM
Re: quant choices for 70B local - by homelabhenry - 10-18-2026, 06:07 PM
Re: quant choices for 70B local - by cacheworks - 10-19-2026, 02:50 AM
Re: quant choices for 70B local - by vrambandit - 10-19-2026, 05:23 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)