+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: what are you all running this week for Phi (/showthread.php?tid=964)
Pages:
1
2
Re: what are you all running this week for Phi - ggufwire22 - 10-16-2026
Good point. That matches what I saw on 7900 XTX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: what are you all running this week for Phi - tensorkite - 10-16-2026
Good point. The pattern around speculative decoding has been pretty similar for me.
Re: what are you all running this week for Phi - cobaltworks25 - 10-16-2026
following this because I'm in the same boat.
Re: what are you all running this week for Phi - marbletrack - 10-16-2026
I tested this tonight. On GB200 with MLX, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: what are you all running this week for Phi - thetaember38 - 10-16-2026
Quote:I tested this tonight. On GB200 with MLX, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collaps
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher.