Inference › Inference Engines › TensorRT-LLM v
quant support gaps for Mixtral

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
quant support gaps for Mixtral
batchcraft Offline
Member
***
Posts: 54
Threads: 8
Joined: Sep 2026
Reputation: 0
#2
10-13-2026, 02:54 AM
Quote:This keeps coming up in conversation so I figured I'd start a thread.

Good point. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Find
Reply


Messages In This Thread
quant support gaps for Mixtral - by syshouse - 10-12-2026, 06:47 PM
Re: quant support gaps for Mixtral - by batchcraft - 10-13-2026, 02:54 AM
Re: quant support gaps for Mixtral - by labline - 10-17-2026, 12:15 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)