+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Prompts / System Prompts (https://www.datasculpting.ai/inference/forumdisplay.php?fid=62)
+--- Thread: system prompt habits that helped: what are you seeing? (/showthread.php?tid=306)
Pages:
1
2
system prompt habits that helped: what are you seeing? - localpatch - 09-17-2026
I'm trying to sanity-check my setup.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling small local stacks.
Not looking for the perfect answer, mostly what worked in real use.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: system prompt habits that helped: what are you seeing? - modrunner27 - 09-17-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: system prompt habits that helped: what are you seeing? - mintstation99 - 09-17-2026
Thanks, that helps. I ended up in about the same place using SGLang. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: system prompt habits that helped: what are you seeing? - thetafield - 09-17-2026
I'd start with the simpler stack.
Re: system prompt habits that helped: what are you seeing? - quiettoken - 09-17-2026
that fixed it for me too.
Re: system prompt habits that helped: what are you seeing? - mlxgarden - 09-17-2026
Good point. The pattern around router policy has been pretty similar for me.
Re: system prompt habits that helped: what are you seeing? - thetafield - 09-17-2026
Quote:Good point. The pattern around router policy has been pretty similar for me.
A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: system prompt habits that helped: what are you seeing? - dockworks23 - 09-20-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: system prompt habits that helped: what are you seeing? - quiettoken - 09-23-2026
My first guess would be this: If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: system prompt habits that helped: what are you seeing? - nanopatch - 09-23-2026
Thanks, that helps. That matches what I saw on GB200.