+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Agents (https://www.datasculpting.ai/inference/forumdisplay.php?fid=44)
+--- Thread: distillation that keeps useful behavior: anyone testing this? (/showthread.php?tid=1484)
Re: distillation that keeps useful behavior: anyone testing this? - mangowire - 10-18-2026
My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: distillation that keeps useful behavior: anyone testing this? - quietstack - 10-18-2026
That matches what I was seeing. The pattern around structured output has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
distillation that keeps useful behavior: anyone testing this? - thetaplane - 10-18-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about token cost.
Simple and boring is a valid answer.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: distillation that keeps useful behavior: anyone testing this? - emberpeak - 10-18-2026
Appreciate the detail. That matches what I saw on 7900 XTX.
Re: distillation that keeps useful behavior: anyone testing this? - llmtrack26 - 10-18-2026
My first guess would be this: On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: distillation that keeps useful behavior: anyone testing this? - amberpatch25 - 10-18-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: distillation that keeps useful behavior: anyone testing this? - torchkite - 10-19-2026
Quote:A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot
Need a little more detail here. Is this local-only or are you routing anything externally?
I'm happy to test one more variant if you want a second data point.
Re: distillation that keeps useful behavior: anyone testing this? - turboyard94 - 10-19-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.