+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Fine-Tuning / LoRA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=54)
+--- Thread: how much orchestration is too much with Ollama (/showthread.php?tid=70)
Pages:
1
2
how much orchestration is too much with Ollama - llm_wren - 08-29-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on mini PC, I ended up with decent performance, but the setup only really made sense once I adjusted around NVFP4.
Nothing scientific here, just practical notes from a real setup.
Re: how much orchestration is too much with Ollama - quantsupply - 08-29-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
Re: how much orchestration is too much with Ollama - torchtrack24 - 08-30-2026
Interesting, I hadn't tried that. I ended up in about the same place using SGLang.
Re: how much orchestration is too much with Ollama - routerunner - 08-30-2026
What worked for me was pretty simple. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: how much orchestration is too much with Ollama - cudacrate - 09-02-2026
Thanks, that helps. The pattern around workstation thermals has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: how much orchestration is too much with Ollama - phasestream53 - 09-02-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: how much orchestration is too much with Ollama - localpatch78 - 09-05-2026
I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
Re: how much orchestration is too much with Ollama - llm_wren - 09-07-2026
Good point. That matches what I saw on dual 3090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: how much orchestration is too much with Ollama - northgrid33 - 09-07-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size? I keep promising myself this is the last rebuild.
Re: how much orchestration is too much with Ollama - cudacrate - 09-07-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.