+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: when TensorRT-LLM clearly wins with MLX (/showthread.php?tid=1145)
when TensorRT-LLM clearly wins with MLX - rasterbench - 10-13-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Kimi, TensorRT-LLM, dual 3090, with a lot of attention on reranking.
If someone wants the exact settings I can clean them up and post them.
Re: when TensorRT-LLM clearly wins with MLX - pixelcrate82 - 10-14-2026
That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: when TensorRT-LLM clearly wins with MLX - northcraft90 - 10-14-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: when TensorRT-LLM clearly wins with MLX - macrotinker - 10-14-2026
Quote:A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: when TensorRT-LLM clearly wins with MLX - kernelmike - 10-14-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: when TensorRT-LLM clearly wins with MLX - syshouse - 10-15-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.
Re: when TensorRT-LLM clearly wins with MLX - kernelmike - 10-15-2026
Quote:My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.