+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Long Context / Memory (https://www.datasculpting.ai/inference/forumdisplay.php?fid=64)
+--- Thread: long context that stayed useful (/showthread.php?tid=955)
long context that stayed useful - torchstation - 10-09-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Kimi, llama.cpp, A6000, with a lot of attention on router policy.
Feel free to tear it apart if you see something dumb.
The part that surprised me was how much the boring details mattered more than the model choice.
Re: long context that stayed useful - torchstation - 10-11-2026
Good point. I ended up in about the same place using llama.cpp. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: long context that stayed useful - tensorriver - 10-11-2026
That matches what I was seeing. That matches what I saw on B200. The logs were somehow calmer than I was.
Re: long context that stayed useful - routecraft - 10-11-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. Once you know the plain path works, then add the faster path back in piece by piece.
Re: long context that stayed useful - quiethouse - 10-11-2026
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: long context that stayed useful - ggufvault - 10-12-2026
What worked for me was pretty simple. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: long context that stayed useful - torchkite - 10-12-2026
A couple things I'd check first. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: long context that stayed useful - turboscope5 - 10-12-2026
Interesting, I hadn't tried that. The pattern around small local stacks has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.