+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Fine-Tuning / LoRA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=54)
+--- Thread: workflow automation with local models with Ollama (/showthread.php?tid=275)
Pages:
1
2
workflow automation with local models with Ollama - llm_wren - 09-15-2026
Maybe I'm missing something obvious.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling HBM constraints.
If you've tested something similar, what did you keep and what did you stop bothering with?
Re: workflow automation with local models with Ollama - llm_wren - 09-16-2026
Quote:Maybe I'm missing something obvious.
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: workflow automation with local models with Ollama - bytegarden73 - 09-16-2026
Thanks, that helps. The pattern around KV cache pressure has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: workflow automation with local models with Ollama - southden - 09-16-2026
Thanks, that helps. The pattern around structured output has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: workflow automation with local models with Ollama - llm_wren - 09-16-2026
Good point. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: workflow automation with local models with Ollama - phasestream53 - 09-17-2026
subscribed, curious where this lands.
Re: workflow automation with local models with Ollama - labline - 09-20-2026
What worked for me was pretty simple. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: workflow automation with local models with Ollama - thetanest - 09-20-2026
What exact setup are you running? Are you testing this on 7900 XTX or something else?
Re: workflow automation with local models with Ollama - llm_wren - 09-20-2026
Thanks, that helps. The pattern around tail latency has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: workflow automation with local models with Ollama - cudaline - 09-21-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.