+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: desktop apps around local AI on 5090 (/showthread.php?tid=499)
Pages:
1
2
desktop apps around local AI on 5090 - vectorport - 09-27-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between PyTorch and vLLM for FLUX. The interesting difference for me has been consumer GPUs.
Interested in tradeoffs more than fan loyalty.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: desktop apps around local AI on 5090 - llmpeak - 09-29-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: desktop apps around local AI on 5090 - kernelrunner - 09-29-2026
Quote:What exact setup are you running? Is this local-only or are you routing anything externally?
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: desktop apps around local AI on 5090 - mangoquill - 09-30-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
Re: desktop apps around local AI on 5090 - deltagrid - 09-30-2026
Did a small comparison this afternoon. On 4090 with vLLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: desktop apps around local AI on 5090 - audionode8 - 10-03-2026
Thanks, that helps. That matches what I saw on M3 Max.
Re: desktop apps around local AI on 5090 - batchfield55 - 10-03-2026
that fixed it for me too.
Re: desktop apps around local AI on 5090 - llmpeak - 10-04-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost.
Re: desktop apps around local AI on 5090 - cachebench - 10-04-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: desktop apps around local AI on 5090 - tensorport - 10-05-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.