+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Long Context / Memory (https://www.datasculpting.ai/inference/forumdisplay.php?fid=64)
+--- Thread: context stuffing mistakes: what are you seeing? (/showthread.php?tid=108)
Pages:
1
2
context stuffing mistakes: what are you seeing? - flintriver50 - 09-02-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate reranking until they actually try to run this stuff for more than one afternoon.
Curious whether that lines up with what other people are seeing.
Re: context stuffing mistakes: what are you seeing? - torchstation - 09-04-2026
Quote:This keeps coming up in conversation so I figured I'd start a thread.
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
Re: context stuffing mistakes: what are you seeing? - llmarch83 - 09-05-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: context stuffing mistakes: what are you seeing? - tensorriver - 09-05-2026
that fixed it for me too.
Re: context stuffing mistakes: what are you seeing? - ggufvault - 09-05-2026
worth checking the driver first.
Re: context stuffing mistakes: what are you seeing? - torchkite - 09-06-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: context stuffing mistakes: what are you seeing? - vrampilot - 09-06-2026
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time. At this point the real bottleneck is my patience.
Re: context stuffing mistakes: what are you seeing? - tensorriver - 09-06-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.
Re: context stuffing mistakes: what are you seeing? - circuitkite - 09-06-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
Re: context stuffing mistakes: what are you seeing? - tinycontext - 09-09-2026
Quote:A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?