when video is still not worth the wait for DeepSeek

[turboscope]

Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about MoE serving.

Would rather learn from your scars than collect my own all at once.

[quantnest]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?

[bytestack]

Maybe, but I had the opposite result. I've seen people blame speculative decoding when the real issue was much more mundane.

[quiettoken]

I'd start with the simpler stack.

[mangovault67]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.

[fastfield4]

Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?

If you post your exact settings, people here can probably narrow it down pretty fast.

[batchscope72]

What worked for me was pretty simple. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[byteyard]

Appreciate the detail. I ended up in about the same place using vLLM.

[southtinker]

Appreciate the detail. I ended up in about the same place using vLLM.

I hit almost the same issue last night.

[macrotinker]

I'd start with the simpler stack.