+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: recipes for a good local stack (/showthread.php?tid=720)
Pages:
1
2
Re: recipes for a good local stack - rocmcrate26 - 10-05-2026
I'd start with the simpler stack. I keep promising myself this is the last rebuild.
Re: recipes for a good local stack - vramstream - 10-05-2026
Interesting, I hadn't tried that. The pattern around NVFP4 has been pretty similar for me.
Re: recipes for a good local stack - llmpeak - 10-05-2026
Quote:Interesting, I hadn't tried that. The pattern around NVFP4 has been pretty similar for me.
My first guess would be this: If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: recipes for a good local stack - rocmcrate26 - 10-05-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: recipes for a good local stack - oxidegrove40 - 10-06-2026
Good point. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: recipes for a good local stack - batchshed - 10-06-2026
That matches what I was seeing. The pattern around tail latency has been pretty similar for me.