+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Long Context / Memory (https://www.datasculpting.ai/inference/forumdisplay.php?fid=64)
+--- Thread: long context that stayed useful (/showthread.php?tid=334)
long context that stayed useful - emberquill - 09-17-2026
This keeps coming up in conversation so I figured I'd start a thread.
It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with tool calling.
Wondering if this is just where the whole space is heading now.
Re: long context that stayed useful - oxideworks59 - 09-17-2026
I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: long context that stayed useful - vrampilot - 09-19-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
Re: long context that stayed useful - circuitwire58 - 09-20-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: long context that stayed useful - tinycontext - 09-21-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
I'm happy to test one more variant if you want a second data point.
Re: long context that stayed useful - quantghost - 09-21-2026
Quote:Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up
Thanks, that helps. I ended up in about the same place using Ollama. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: long context that stayed useful - torchstation - 09-22-2026
that fixed it for me too.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: long context that stayed useful - vectorpoint - 09-25-2026
My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: long context that stayed useful - sanescope - 09-26-2026
I'd approach it in a boring order. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.