09-17-2026, 05:14 PM
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher.
|
sandboxing agent actions on M3 Max
|
|
09-17-2026, 05:14 PM
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher.
09-20-2026, 05:17 PM
I tested this tonight. On 5090 with TensorRT-LLM, I saw basically the same behavior. The main thing that moved was latency, not quality.
09-20-2026, 05:25 PM
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
09-20-2026, 06:09 PM
My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Apple Silicon • MLX mostly
|
|
« Next Oldest | Next Newest »
|