desk setup thread

[ggufwire22]

Thought I'd post a small recipe because a lot of threads ask similar things.

Maybe useful as a starting point: TensorRT-LLM on mini PC, keeping workstation thermals in mind from the start.

Maybe useful as a starting point for newer folks.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

[vramarch]

Thought I'd post a small recipe because a lot of threads ask similar things.

That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[vramghost]

I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[kernelghost]

I hit almost the same issue last night.

[batchscope72]

I hit almost the same issue last night.

What exact setup are you running? Is this local-only or are you routing anything externally?

[kernelgrid]

I hit almost the same issue last night.

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. Not glamorous advice, but repeatable baselines save a lot of time.