Posts: 38
Threads: 7
Joined: Oct 2026
Reputation:
0
Curious what people here are seeing.
I'm mostly working with DeepSeek in MLX on GB200. The part I'm trying to understand is prefix caching.
If you've tested something similar, what did you keep and what did you stop bothering with?
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Posts: 37
Threads: 1
Joined: Oct 2026
Reputation:
0
Good point. I ended up in about the same place using MLX. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
I'm happy to test one more variant if you want a second data point.
Posts: 37
Threads: 4
Joined: Sep 2026
Reputation:
0
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at embeddings.
Posts: 25
Threads: 2
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at NVFP4. Once you know the plain path works, then add the faster path back in piece by piece.
Posts: 82
Threads: 7
Joined: Sep 2026
Reputation:
0
Ran a quick check on my box. On Threadripper box with Ollama, I saw basically the same behavior. I expected the model choice to matter more than it did.
Posts: 112
Threads: 12
Joined: Sep 2026
Reputation:
0
subscribed, curious where this lands.
context windows are not a product strategy
Posts: 82
Threads: 7
Joined: Sep 2026
Reputation:
0
A couple questions before I guess too hard. Are you testing this on dual 3090 or something else?
Posts: 38
Threads: 7
Joined: Oct 2026
Reputation:
0
Appreciate the detail. The pattern around MoE serving has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
