+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: VRAM bottleneck on consumer cards on M3 Max (/showthread.php?tid=134)
VRAM bottleneck on consumer cards on M3 Max - sparrowhall - 09-05-2026
Maybe I'm missing something obvious.
My current setup is PyTorch with Threadripper box, and I keep running into VRAM headroom.
Would appreciate practical advice rather than theory.
Re: VRAM bottleneck on consumer cards on M3 Max - thetarunner - 09-05-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: VRAM bottleneck on consumer cards on M3 Max - torchshed - 09-05-2026
Need a little more detail here. Is this local-only or are you routing anything externally?