Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
adapter stacking for DeepSeek
#6
Quote:[quote]I think the bottleneck is somewhere else. I get why that seems right, but on Threadripper box the bottleneck felt elsewhere. Not saying you're

Interesting, I hadn't tried that. The pattern around INT4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

If you post your exact settings, people here can probably narrow it down pretty fast.
throughput matters, but tail latency matters too
Reply


Messages In This Thread
adapter stacking for DeepSeek - by cacheline - 10-08-2026, 02:25 AM
Re: adapter stacking for DeepSeek - by cudascope - 10-08-2026, 06:46 AM
Re: adapter stacking for DeepSeek - by batchsize - 10-11-2026, 07:23 AM
Re: adapter stacking for DeepSeek - by batchsize - 10-17-2026, 07:26 PM
Re: adapter stacking for DeepSeek - by batchsize - 10-13-2026, 10:03 AM
Re: adapter stacking for DeepSeek - by cacheline - 10-16-2026, 10:45 AM
Re: adapter stacking for DeepSeek - by kernelarch - 10-16-2026, 11:56 AM
Re: adapter stacking for DeepSeek - by quiethouse - 10-17-2026, 12:03 PM
Re: adapter stacking for DeepSeek - by batchsize - 10-17-2026, 02:22 PM
Re: adapter stacking for DeepSeek - by cudascope - 10-17-2026, 04:36 PM
Re: adapter stacking for DeepSeek - by cudascope - 10-17-2026, 07:09 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)