Posts: 40
Threads: 4
Joined: Sep 2026
Reputation:
0
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of LM Studio and Threadripper box felt surprisingly usable once I tuned around VRAM headroom.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
Posts: 52
Threads: 4
Joined: Sep 2026
Reputation:
0
What worked for me was pretty simple. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Posts: 128
Threads: 12
Joined: Sep 2026
Reputation:
0
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4. Once you know the plain path works, then add the faster path back in piece by piece.
rack in the garage • local models everywhere
Posts: 37
Threads: 6
Joined: Oct 2026
Reputation:
0
that fixed it for me too.
Posts: 26
Threads: 3
Joined: Aug 2026
Reputation:
0
Good point. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Posts: 59
Threads: 8
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Posts: 34
Threads: 4
Joined: Aug 2026
Reputation:
0
Tried to reproduce it just now. On mini PC with vLLM, I saw basically the same behavior.
small models can still be good
Posts: 35
Threads: 3
Joined: Sep 2026
Reputation:
0
Appreciate the detail. The pattern around NVFP4 has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Posts: 45
Threads: 8
Joined: Sep 2026
Reputation:
0
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
team red, team patient
Posts: 37
Threads: 6
Joined: Oct 2026
Reputation:
0
I'd approach it in a boring order. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
