+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: small win thread on 5090 (/showthread.php?tid=348)
Pages:
1
2
small win thread on 5090 - quiettoken - 09-18-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: MLX on 5090, keeping long context in mind from the start.
If someone wants the exact settings I can clean them up and post them.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: small win thread on 5090 - quiettoken - 09-19-2026
I think the bottleneck is somewhere else. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: small win thread on 5090 - cudacrate - 09-19-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: small win thread on 5090 - latentscope35 - 09-20-2026
I think the bottleneck is somewhere else. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: small win thread on 5090 - cacheplane98 - 09-21-2026
I hit almost the same issue last night.
Re: small win thread on 5090 - marbletrack - 09-21-2026
Thanks, that helps. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: small win thread on 5090 - ggufwire22 - 09-22-2026
Interesting, I hadn't tried that. That matches what I saw on mini PC. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: small win thread on 5090 - cacheplane98 - 09-22-2026
I'd approach it in a boring order. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: small win thread on 5090 - quiettoken - 09-22-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: small win thread on 5090 - plaingarden18 - 09-22-2026
Thanks, that helps. I don't think your conclusion is far off.