+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: working config for 32k context (/showthread.php?tid=722)
Pages:
1
2
Re: working config for 32k context - rocmriley - 10-06-2026
That matches what I was seeing. I ended up in about the same place using Ollama. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: working config for 32k context - benchplease - 10-06-2026
Quote:That matches what I was seeing. I ended up in about the same place using Ollama. The biggest change for me was simplifying the stack instead of adding
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?