recipes for a good local stack

[rocmcrate26]

I'd start with the simpler stack. I keep promising myself this is the last rebuild.

[vramstream]

Interesting, I hadn't tried that. The pattern around NVFP4 has been pretty similar for me.

[llmpeak]

Interesting, I hadn't tried that. The pattern around NVFP4 has been pretty similar for me.

My first guess would be this: If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.

[rocmcrate26]

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.

[oxidegrove40]

Good point. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[batchshed]

That matches what I was seeing. The pattern around tail latency has been pretty similar for me.