distillation that keeps useful behavior: anyone testing this?

[mangowire]

My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[quietstack]

That matches what I was seeing. The pattern around structured output has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[thetaplane]

Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about token cost.

Simple and boring is a valid answer.

I care more about stability than squeezing the last possible tokens/sec out of it.

[emberpeak]

Appreciate the detail. That matches what I saw on 7900 XTX.

[llmtrack26]

My first guess would be this: On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[amberpatch25]

Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.

[torchkite]

A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot

Need a little more detail here. Is this local-only or are you routing anything externally?

I'm happy to test one more variant if you want a second data point.

[turboyard94]

A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.