+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: SBCs / Mini PCs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=74)
+--- Thread: Jetson still a thing (/showthread.php?tid=1197)
Pages:
1
2
Jetson still a thing - batchfield55 - 10-14-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of LM Studio and Threadripper box felt surprisingly usable once I tuned around VRAM headroom.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
Re: Jetson still a thing - cobalttinker - 10-16-2026
What worked for me was pretty simple. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Jetson still a thing - homelabhenry - 10-16-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Jetson still a thing - phasespark - 10-16-2026
that fixed it for me too.
Re: Jetson still a thing - oxiderunner - 10-16-2026
Good point. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Jetson still a thing - modbench - 10-17-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: Jetson still a thing - quantsupply - 10-18-2026
Tried to reproduce it just now. On mini PC with vLLM, I saw basically the same behavior.
Re: Jetson still a thing - northquill - 10-18-2026
Appreciate the detail. The pattern around NVFP4 has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Jetson still a thing - mintquill - 10-18-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: Jetson still a thing - phasespark - 10-19-2026
I'd approach it in a boring order. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.