+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Hosted (https://www.datasculpting.ai/inference/forumdisplay.php?fid=50)
+---- Thread: high-context local setup (/showthread.php?tid=1431)
Pages:
1
2
Re: high-context local setup - latentgarden - 10-19-2026
I'd approach it in a boring order. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: high-context local setup - mangopath4 - 10-19-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: high-context local setup - bytelab - 10-19-2026
I'd approach it in a boring order. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: high-context local setup - quietcraft - 10-19-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: high-context local setup - embershed99 - 10-19-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: high-context local setup - deepseeked - 10-19-2026
same here on my side.