non-AI thing you built recently for DeepSeek

[promptfield]

Has anyone else run into this?

I'm mostly working with DeepSeek in vLLM on GB200. The part I'm trying to understand is speculative decoding.

Would appreciate practical advice rather than theory.

I care more about stability than squeezing the last possible tokens/sec out of it.

[turboscope]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[batchfield55]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end

Maybe, but I had the opposite result. I've seen people blame prefill/decode split when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.

[rocmmixer]

Thanks, that helps. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[sparrowhall]

Thanks, that helps. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow fe

I'd approach it in a boring order. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.

[nanobrook]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[flintriver50]

What worked for me was pretty simple. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious.