odd little inference appliances with TensorRT-LLM

[orbitline]

I'm trying to sanity-check my setup.

My current setup is LM Studio with dual 3090, and I keep running into prefill/decode split.

Not looking for the perfect answer, mostly what worked in real use.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

[routearch]

Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.