quiet dual-GPU build advice

[localcraft17]

Did a bit of testing and thought I'd post the result here.

I tried this with Mistral using TensorRT-LLM on mini PC. The headline result was better than I expected, but speculative decoding still ended up being the limiting factor.

Happy to post more details if anyone wants them.

[edgequill]

Did a bit of testing and thought I'd post the result here.

What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching. Not glamorous advice, but repeatable baselines save a lot of time.

[bytepilot]

That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.