+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: what are you all running this week (/showthread.php?tid=1405)
Pages:
1
2
what are you all running this week - rasterpoint - 10-17-2026
This keeps coming up in conversation so I figured I'd start a thread.
It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with HBM constraints.
Curious whether that lines up with what other people are seeing.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: what are you all running this week - mangofield33 - 10-17-2026
What exact setup are you running? Are you testing this on GB200 or something else?
Re: what are you all running this week - southpatch - 10-17-2026
Appreciate the detail. The pattern around tool calling has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: what are you all running this week - cacheplane98 - 10-18-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there.
Re: what are you all running this week - vectorfield - 10-18-2026
A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: what are you all running this week - vramarch - 10-19-2026
Appreciate the detail. I ended up in about the same place using PyTorch. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable. At this point the real bottleneck is my patience.
Re: what are you all running this week - rocmrunner40 - 10-19-2026
Quote:A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of
What worked for me was pretty simple. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: what are you all running this week - quiettoken - 10-19-2026
Good point. The pattern around long context has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: what are you all running this week - plaingarden18 - 10-19-2026
That matches what I was seeing. That matches what I saw on M3 Max.
Re: what are you all running this week - marbletrack - 10-19-2026
A couple things I'd check first. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.