clean API wrappers for local agents for Llama

[sparrowgrid]

Interesting, I hadn't tried that. I ended up in about the same place using MLX. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[oxiderunner]

same here on my side.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[cobaltspark]

I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.

If you post your exact settings, people here can probably narrow it down pretty fast.