+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: Off Topic (https://www.datasculpting.ai/inference/forumdisplay.php?fid=27)
+--- Thread: what music do you benchmark to (/showthread.php?tid=354)
Pages:
1
2
what music do you benchmark to - deltastation - 09-19-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on M3 Max, I ended up with decent performance, but the setup only really made sense once I adjusted around prefill/decode split.
Happy to post more details if anyone wants them.
Re: what music do you benchmark to - rasterhall - 09-21-2026
That matches what I was seeing. The pattern around KV cache pressure has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: what music do you benchmark to - signaltrack44 - 09-21-2026
Thanks, that helps. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: what music do you benchmark to - cozygrid12 - 09-21-2026
I hit almost the same issue last night.
Re: what music do you benchmark to - vectorpoint - 09-22-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.
Re: what music do you benchmark to - kernelmike - 09-22-2026
Quote:I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: what music do you benchmark to - cobalttinker - 09-22-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. Once you know the plain path works, then add the faster path back in piece by piece.
Re: what music do you benchmark to - sparrowgrid - 09-25-2026
I hit almost the same issue last night.
Re: what music do you benchmark to - edgeriver - 09-25-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: what music do you benchmark to - paperport64 - 09-26-2026
Quote:Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.