tool calling reliability with PyTorch

[tensorport]

I tested this tonight. On B200 with Ollama, I saw basically the same behavior. The main thing that moved was latency, not quality.

Good point. I ended up in about the same place using llama.cpp.

[quantsupply]

What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.

[batchfield55]

What worked for me was pretty simple. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.

A couple questions before I guess too hard. What model, what runtime, and roughly what context size?

[llmpeak]

worth checking the driver first.

[macropilot]

Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.