+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: when TensorRT-LLM clearly wins with vLLM (/showthread.php?tid=503)
when TensorRT-LLM clearly wins with vLLM - kernelmike - 09-27-2026
Wanted to compare notes with people doing similar things.
I'm mostly working with Mixtral in MLX on 5090. The part I'm trying to understand is tokens/sec.
If you've tested something similar, what did you keep and what did you stop bothering with?
This is mostly for local work rather than public serving.