Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
NVFP4 / INT4 tradeoffs
#1
Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about VRAM headroom.

Simple and boring is a valid answer.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

The part that surprised me was how much the boring details mattered more than the model choice.
team red, team patient
Reply
#2
A couple things I'd check first. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.
team red, team patient
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)