+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Long Context / Memory (https://www.datasculpting.ai/inference/forumdisplay.php?fid=64)
+--- Thread: long context that stayed useful with MLX (/showthread.php?tid=1191)
long context that stayed useful with MLX - promptfield - 10-13-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between SGLang and MLX for Llama. The interesting difference for me has been embeddings.
What would you pick if you had to live with it for a few months?
This is mostly for local work rather than public serving.
Re: long context that stayed useful with MLX - mellowsupply - 10-13-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: long context that stayed useful with MLX - turboscope5 - 10-13-2026
What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: long context that stayed useful with MLX - fastfield4 - 10-14-2026
I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: long context that stayed useful with MLX - bytesupply76 - 10-14-2026
Thanks, that helps. The pattern around INT4 has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: long context that stayed useful with MLX - papermixer4 - 10-14-2026
That matches what I was seeing. The pattern around tail latency has been pretty similar for me. I keep promising myself this is the last rebuild.
Re: long context that stayed useful with MLX - sanescope - 10-14-2026
Thanks, that helps. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: long context that stayed useful with MLX - southspark - 10-14-2026
Ran a quick check on my box. On M3 Max with PyTorch, I saw basically the same behavior.