Posts: 37
Threads: 6
Joined: Oct 2026
Reputation:
0
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate long context until they actually try to run this stuff for more than one afternoon.
Curious whether that lines up with what other people are seeing.
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation:
0
I'd start with the simpler stack.
throughput matters, but tail latency matters too
Posts: 49
Threads: 5
Joined: Sep 2026
Reputation:
0
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Posts: 341
Threads: 51
Joined: Sep 2026
Reputation:
0
That matches what I was seeing. The pattern around NVFP4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
measure twice, benchmark three times
Posts: 112
Threads: 12
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
context windows are not a product strategy
Posts: 36
Threads: 3
Joined: Oct 2026
Reputation:
0
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.
still benchmarking everything
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
throughput matters, but tail latency matters too
Posts: 30
Threads: 6
Joined: Sep 2026
Reputation:
0
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
learning in public
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation:
0
Quote:Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest o
worth checking the driver first.
If you post your exact settings, people here can probably narrow it down pretty fast.
throughput matters, but tail latency matters too
Posts: 59
Threads: 7
Joined: Oct 2026
Reputation:
0
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
learning in public
