+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: industry move that matters for local users for Llama (/showthread.php?tid=1397)
Pages:
1
2
Re: industry move that matters for local users for Llama - llmpeak - 10-19-2026
Appreciate the detail. I ended up in about the same place using MLX.
Re: industry move that matters for local users for Llama - phaseshed - 10-19-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding.
Re: industry move that matters for local users for Llama - bytestack - 10-19-2026
that fixed it for me too.
Re: industry move that matters for local users for Llama - modbench - 10-19-2026
Tried to reproduce it just now. On M3 Max with Ollama, I saw basically the same behavior. I expected the model choice to matter more than it did.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: industry move that matters for local users for Llama - modbench - 10-19-2026
Not sure I agree there. I've seen people blame batch size when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
I'm happy to test one more variant if you want a second data point.
Re: industry move that matters for local users for Llama - southpatch - 10-19-2026
Interesting, I hadn't tried that. I ended up in about the same place using PyTorch. The biggest change for me was simplifying the stack instead of adding one more optimization layer.