+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Embedding / Reranking Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=53)
+--- Thread: tool calling reliability (/showthread.php?tid=377)
tool calling reliability - batchpeak - 09-20-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once batch size becomes important.
Trying to avoid rebuilding the whole thing twice.
This is mostly for local work rather than public serving.
Re: tool calling reliability - mangowire - 09-22-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: tool calling reliability - tensorriver - 09-23-2026
I tested this tonight. On 4090 with Ollama, I saw basically the same behavior.
Re: tool calling reliability - llm_wren - 09-23-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at router policy. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: tool calling reliability - audiopath66 - 09-23-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4. Not glamorous advice, but repeatable baselines save a lot of time.
Re: tool calling reliability - localpatch78 - 09-24-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: tool calling reliability - cacheline - 09-24-2026
Quote:My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. A lot of these threads end up
same here on my side.
Re: tool calling reliability - griffinloom66 - 09-26-2026
Tried to reproduce it just now. On mini PC with vLLM, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: tool calling reliability - torchsmith25 - 09-26-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Re: tool calling reliability - sparrowworks88 - 09-26-2026
A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives.