Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
prefill bottlenecks on longer prompts on A6000
#4
That matches what I was seeing. I ended up in about the same place using PyTorch. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

If you post your exact settings, people here can probably narrow it down pretty fast.
small models can still be good
Reply


Messages In This Thread
Re: prefill bottlenecks on longer prompts on A6000 - by kernelrunner - 09-28-2026, 01:25 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)