+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: working config for 32k context for Llama (/showthread.php?tid=27)
Pages:
1
2
Re: working config for 32k context for Llama - rocmhouse90 - 08-30-2026
Quote:Appreciate the detail. That matches what I saw on 7900 XTX.
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: working config for 32k context for Llama - relayforge - 08-31-2026
Quote:Quote:Appreciate the detail. That matches what I saw on 7900 XTX.
Interesting, I hadn't tried that. I ended up in about the same place using vLLM.
Re: working config for 32k context for Llama - cozysmith - 09-02-2026
Good point. I ended up in about the same place using llama.cpp.
Re: working config for 32k context for Llama - llmpeak - 09-05-2026
Quote:Good point. I ended up in about the same place using llama.cpp.
Need a little more detail here. Is this local-only or are you routing anything externally?
Re: working config for 32k context for Llama - bytehall - 09-05-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.