Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
VRAM bottleneck on consumer cards for Llama
#8
That matches what I was seeing. The pattern around HBM constraints has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

If you post your exact settings, people here can probably narrow it down pretty fast.
throughput matters, but tail latency matters too
Reply


Messages In This Thread
Re: VRAM bottleneck on consumer cards for Llama - by batchsize - 10-18-2026, 10:59 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)