+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: NVFP4 / INT4 tradeoffs: setup advice (/showthread.php?tid=548)
Pages:
1
2
Re: NVFP4 / INT4 tradeoffs: setup advice - benchplease - 10-07-2026
Need a little more detail here. Is this local-only or are you routing anything externally?
Re: NVFP4 / INT4 tradeoffs: setup advice - mellowwire98 - 10-08-2026
That matches what I was seeing. I ended up in about the same place using TensorRT-LLM.
Re: NVFP4 / INT4 tradeoffs: setup advice - vramforge - 10-09-2026
Quote:That matches what I was seeing. I ended up in about the same place using TensorRT-LLM.
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: NVFP4 / INT4 tradeoffs: setup advice - routeline23 - 10-09-2026
Quote:Quote:That matches what I was seeing. I ended up in about the same place using TensorRT-LLM.
I'd approach it in a boring order. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: NVFP4 / INT4 tradeoffs: setup advice - macroforge - 10-09-2026
Appreciate the detail. The pattern around KV cache pressure has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.