Threadripper build for inference: setup advice

[macroforge]

Thanks, that helps. I ended up in about the same place using SGLang. The biggest change for me was simplifying the stack instead of adding one more optimization layer.