+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: working config for 32k context (/showthread.php?tid=987)
Pages:
1
2
Re: working config for 32k context - vramstream - 10-16-2026
Thanks, that helps. I ended up in about the same place using MLX.
Re: working config for 32k context - vectorport - 10-17-2026
I'd start with the simpler stack.
Re: working config for 32k context - deltagrid - 10-18-2026
I'd start with the simpler stack.
Re: working config for 32k context - audioriver - 10-18-2026
I tested this tonight. On GB200 with PyTorch, I saw basically the same behavior. I expected the model choice to matter more than it did.