odd little inference appliances with TensorRT-LLM

[orbitline]

I'm trying to sanity-check my setup.

My current setup is LM Studio with dual 3090, and I keep running into prefill/decode split.

Not looking for the perfect answer, mostly what worked in real use.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.