NVFP4 / INT4 tradeoffs: setup advice

[benchplease]

Need a little more detail here. Is this local-only or are you routing anything externally?

[mellowwire98]

That matches what I was seeing. I ended up in about the same place using TensorRT-LLM.

[vramforge]

That matches what I was seeing. I ended up in about the same place using TensorRT-LLM.

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[routeline23]

That matches what I was seeing. I ended up in about the same place using TensorRT-LLM.

I'd approach it in a boring order. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[macroforge]

Appreciate the detail. The pattern around KV cache pressure has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.