+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Workflows / Automation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=47)
+--- Thread: DeepSeek V4.1 impressions with PyTorch (/showthread.php?tid=608)
Pages:
1
2
DeepSeek V4.1 impressions with PyTorch - routecraft - 10-01-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: DeepSeek, SGLang, M3 Max, with a lot of attention on batch size.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
Re: DeepSeek V4.1 impressions with PyTorch - routecraft - 10-01-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tail latency. Once you know the plain path works, then add the faster path back in piece by piece.
Re: DeepSeek V4.1 impressions with PyTorch - latentgarden - 10-01-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tail latency. Not glamorous advice, but repeatable baselines save a lot of time.
Re: DeepSeek V4.1 impressions with PyTorch - routecraft - 10-01-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: DeepSeek V4.1 impressions with PyTorch - macrolab - 10-01-2026
Quote:Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it
That matches what I was seeing. That matches what I saw on M3 Max. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: DeepSeek V4.1 impressions with PyTorch - routecraft - 10-02-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: DeepSeek V4.1 impressions with PyTorch - routecraft - 10-03-2026
Need a little more detail here. Are you testing this on Threadripper box or something else?
Re: DeepSeek V4.1 impressions with PyTorch - rocmriley - 10-03-2026
following this because I'm in the same boat.
Re: DeepSeek V4.1 impressions with PyTorch - routecraft - 10-03-2026
I'd start with the simpler stack.
Re: DeepSeek V4.1 impressions with PyTorch - torchsupply99 - 10-03-2026
What worked for me was pretty simple. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.