+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: Off Topic (https://www.datasculpting.ai/inference/forumdisplay.php?fid=27)
+--- Thread: desk setup thread (/showthread.php?tid=1072)
desk setup thread - ggufwire22 - 10-11-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: TensorRT-LLM on mini PC, keeping workstation thermals in mind from the start.
Maybe useful as a starting point for newer folks.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: desk setup thread - vramarch - 10-12-2026
Quote:Thought I'd post a small recipe because a lot of threads ask similar things.
That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: desk setup thread - vramghost - 10-12-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: desk setup thread - kernelghost - 10-12-2026
I hit almost the same issue last night.
Re: desk setup thread - batchscope72 - 10-12-2026
Quote:I hit almost the same issue last night.
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: desk setup thread - kernelgrid - 10-12-2026
Quote:Quote:I hit almost the same issue last night.
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. Not glamorous advice, but repeatable baselines save a lot of time.