10-14-2026, 03:03 PM
Has anyone else run into this?
My current setup is TensorRT-LLM with Threadripper box, and I keep running into VRAM headroom.
Not looking for the perfect answer, mostly what worked in real use.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
My current setup is TensorRT-LLM with Threadripper box, and I keep running into VRAM headroom.
Not looking for the perfect answer, mostly what worked in real use.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.