+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Announcements (https://www.datasculpting.ai/inference/forumdisplay.php?fid=5)
+--- Forum: Community Announcements (https://www.datasculpting.ai/inference/forumdisplay.php?fid=7)
+--- Thread: Forum structure feedback thread on 4090 (/showthread.php?tid=344)
Pages:
1
2
Forum structure feedback thread on 4090 - sparrowworks88 - 09-19-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of LM Studio and M3 Max felt surprisingly usable once I tuned around continuous batching.
If someone wants the exact settings I can clean them up and post them.
Re: Forum structure feedback thread on 4090 - audionode8 - 09-19-2026
subscribed, curious where this lands.
Re: Forum structure feedback thread on 4090 - sparrowhall - 09-22-2026
I tested this tonight. On A6000 with TensorRT-LLM, I saw basically the same behavior.
Re: Forum structure feedback thread on 4090 - tinycontext - 09-24-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: Forum structure feedback thread on 4090 - sanevault - 09-27-2026
Quote:Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: Forum structure feedback thread on 4090 - rocmrunner40 - 09-27-2026
Quote:[quote]Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure sho
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Forum structure feedback thread on 4090 - northcraft90 - 09-27-2026
that fixed it for me too.
Re: Forum structure feedback thread on 4090 - rocmplane - 09-27-2026
What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Forum structure feedback thread on 4090 - relayforge - 09-27-2026
Good point. That matches what I saw on dual 3090.
Re: Forum structure feedback thread on 4090 - cudapoint - 09-27-2026
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.