working config for 32k context

[rocmriley]

That matches what I was seeing. I ended up in about the same place using Ollama. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[benchplease]

That matches what I was seeing. I ended up in about the same place using Ollama. The biggest change for me was simplifying the stack instead of adding

A couple questions before I guess too hard. What model, what runtime, and roughly what context size?