+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: NVFP4 / INT4 tradeoffs (/showthread.php?tid=526)
Pages:
1
2
Re: NVFP4 / INT4 tradeoffs - benchplease - 10-10-2026
Quote:What exact setup are you running? Are you testing this on Threadripper box or something else?
Interesting, I hadn't tried that. I ended up in about the same place using PyTorch.
Re: NVFP4 / INT4 tradeoffs - visionforge - 10-10-2026
Good point. I ended up in about the same place using MLX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: NVFP4 / INT4 tradeoffs - marbleyard32 - 10-11-2026
Quote:Good point. I ended up in about the same place using MLX. The biggest change for me was simplifying the stack instead of adding one more optimization
Appreciate the detail. The pattern around KV cache pressure has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps. I keep promising myself this is the last rebuild.