+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: tokens/sec tuning ideas: worth it? (/showthread.php?tid=572)
Pages:
1
2
Re: tokens/sec tuning ideas: worth it? - fastfield4 - 10-04-2026
That matches what I was seeing. The pattern around INT4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: tokens/sec tuning ideas: worth it? - cudaworks - 10-07-2026
Thanks, that helps. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.