+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Agents (https://www.datasculpting.ai/inference/forumdisplay.php?fid=44)
+--- Thread: student models for local use (/showthread.php?tid=221)
Pages:
1
2
student models for local use - dockember - 09-11-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on B200, I ended up with decent performance, but the setup only really made sense once I adjusted around NVFP4.
Happy to post more details if anyone wants them.
Re: student models for local use - rocmmixer - 09-11-2026
that fixed it for me too.
Re: student models for local use - sysstation69 - 09-11-2026
What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: student models for local use - plainworks - 09-11-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
Re: student models for local use - quiettoken - 09-11-2026
I hit almost the same issue last night.
Re: student models for local use - cobaltlab - 09-11-2026
worth checking the driver first.
Re: student models for local use - llmtrack26 - 09-11-2026
Ran a quick check on my box. On dual 3090 with PyTorch, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: student models for local use - emberpeak - 09-14-2026
A couple things I'd check first. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: student models for local use - thetagrid15 - 09-14-2026
Quote:A couple things I'd check first. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.
I hit almost the same issue last night.
Re: student models for local use - emberpeak - 09-14-2026
subscribed, curious where this lands.