+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: little utility you keep installed for Qwen (/showthread.php?tid=1357)
Pages:
1
2
little utility you keep installed for Qwen - relayforge - 10-16-2026
Has anyone else run into this?
My current setup is LM Studio with dual 3090, and I keep running into consumer GPUs.
If you've tested something similar, what did you keep and what did you stop bothering with?
Re: little utility you keep installed for Qwen - rasterbench - 10-17-2026
Quote:Has anyone else run into this?
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: little utility you keep installed for Qwen - rocmriley - 10-17-2026
Interesting, I hadn't tried that. I ended up in about the same place using MLX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: little utility you keep installed for Qwen - mlxgarden - 10-18-2026
Thanks, that helps. The pattern around tokens/sec has been pretty similar for me.
Re: little utility you keep installed for Qwen - rasterbench - 10-18-2026
Quote:Thanks, that helps. The pattern around tokens/sec has been pretty similar for me.
A couple things I'd check first. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: little utility you keep installed for Qwen - tinycontext - 10-19-2026
I tested this tonight. On mini PC with Ollama, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: little utility you keep installed for Qwen - kernelmike - 10-19-2026
subscribed, curious where this lands.
Re: little utility you keep installed for Qwen - batchshed - 10-19-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.
Re: little utility you keep installed for Qwen - visionforge - 10-19-2026
I'd start with the simpler stack.
Re: little utility you keep installed for Qwen - kernelgrid - 10-19-2026
A couple things I'd check first. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.