+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: best local runtime for mixed models with TensorRT-LLM (/showthread.php?tid=21)
best local runtime for mixed models with TensorRT-LLM - latentscope35 - 08-24-2026
Did a bit of testing and thought I'd post the result here.
Short version: PyTorch behaved well for me, but only after I stopped ignoring NVFP4.
Curious if anyone else landed in the same place.
Re: best local runtime for mixed models with TensorRT-LLM - mellowwire98 - 08-26-2026
Interesting, I hadn't tried that. The pattern around small local stacks has been pretty similar for me.