Posts: 69
Threads: 3
Joined: Sep 2026
Reputation:
0
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of PyTorch and M3 Max felt surprisingly usable once I tuned around prefill/decode split.
If someone wants the exact settings I can clean them up and post them.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
still benchmarking everything
Posts: 21
Threads: 2
Joined: Sep 2026
Reputation:
0
Ran a quick check on my box. On B200 with TensorRT-LLM, I saw basically the same behavior. I expected the model choice to matter more than it did. At this point the real bottleneck is my patience.
Posts: 48
Threads: 9
Joined: Sep 2026
Reputation:
0
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.
Posts: 184
Threads: 23
Joined: Sep 2026
Reputation:
0
Interesting, I hadn't tried that. The pattern around workstation thermals has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Mostly CUDA, sometimes wrong, usually measured
Posts: 83
Threads: 6
Joined: Sep 2026
Reputation:
0
I don't think that's the whole story. I've seen people blame tokens/sec when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Posts: 45
Threads: 6
Joined: Sep 2026
Reputation:
0
Good point. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Posts: 34
Threads: 2
Joined: Sep 2026
Reputation:
0
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
small models can still be good
Posts: 64
Threads: 12
Joined: Sep 2026
Reputation:
0
Appreciate the detail. I don't think your conclusion is far off.
learning in public
Posts: 60
Threads: 5
Joined: Sep 2026
Reputation:
0
worth checking the driver first.
Posts: 48
Threads: 9
Joined: Sep 2026
Reputation:
0
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
