odd little inference appliances with TensorRT-LLM - Printable Version +- Inference (https://www.datasculpting.ai/inference) +-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21) +--- Forum: AI Devices (https://www.datasculpting.ai/inference/forumdisplay.php?fid=75) +--- Thread: odd little inference appliances with TensorRT-LLM (/showthread.php?tid=1068)

odd little inference appliances with TensorRT-LLM - orbitline - 10-11-2026 I'm trying to sanity-check my setup. My current setup is LM Studio with dual 3090, and I keep running into prefill/decode split. Not looking for the perfect answer, mostly what worked in real use. My use case is pretty ordinary: local chat, some document work, and occasional batch jobs. Re: odd little inference appliances with TensorRT-LLM - routearch - 10-11-2026 Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.