+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: serving on Blackwell (/showthread.php?tid=1169)
serving on Blackwell - kernelgrid - 10-14-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Kimi using SGLang on Threadripper box. The headline result was better than I expected, but embeddings still ended up being the limiting factor.
Curious if anyone else landed in the same place.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: serving on Blackwell - macrotinker - 10-15-2026
I don't think that's the whole story. I've seen people blame NVFP4 when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: serving on Blackwell - rocmriley - 10-15-2026
Interesting, I hadn't tried that. That matches what I saw on Threadripper box. The biggest change for me was simplifying the stack instead of adding one more optimization layer.