tool-calling security concerns: worth it?

[benchplease]

Did a bit of testing and thought I'd post the result here.

I tried this with Mistral using LM Studio on GB200. The headline result was better than I expected, but VRAM headroom still ended up being the limiting factor.

Happy to post more details if anyone wants them.

[rasterhall]

What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?

[lablab]

Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.