+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=65)
+--- Thread: benchmark result I don't trust (/showthread.php?tid=629)
benchmark result I don't trust - mellowwire98 - 10-01-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Llama using PyTorch on M3 Max. The headline result was better than I expected, but consumer GPUs still ended up being the limiting factor.
Nothing scientific here, just practical notes from a real setup.
Re: benchmark result I don't trust - sparrowworks88 - 10-02-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. I keep promising myself this is the last rebuild.
Re: benchmark result I don't trust - quiethouse - 10-02-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: benchmark result I don't trust - vectorfield - 10-03-2026
Good point. The pattern around NVFP4 has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: benchmark result I don't trust - localtinker - 10-03-2026
worth checking the driver first.
Re: benchmark result I don't trust - llmarch83 - 10-03-2026
following this because I'm in the same boat.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: benchmark result I don't trust - dockridge - 10-04-2026
worth checking the driver first.