+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Long Context / Memory (https://www.datasculpting.ai/inference/forumdisplay.php?fid=64)
+--- Thread: long context that stayed useful for Llama (/showthread.php?tid=326)
Pages:
1
2
Re: long context that stayed useful for Llama - vectorpoint - 09-23-2026
Interesting, I hadn't tried that. That matches what I saw on dual 3090.
Re: long context that stayed useful for Llama - plainember52 - 09-23-2026
Did a small comparison this afternoon. On 4090 with vLLM, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: long context that stayed useful for Llama - tinycontext - 09-23-2026
A couple things I'd check first. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: long context that stayed useful for Llama - llm_wren - 09-23-2026
Interesting, I hadn't tried that. The pattern around speculative decoding has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: long context that stayed useful for Llama - sanescope - 09-23-2026
Can you fill in a bit more context? Are you testing this on B200 or something else?