+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Fine-Tuning / LoRA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=54)
+--- Thread: how much orchestration is too much: early impressions (/showthread.php?tid=744)
how much orchestration is too much: early impressions - llm_wren - 10-04-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between Ollama and Ollama for Kimi. The interesting difference for me has been router policy.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: how much orchestration is too much: early impressions - llm_wren - 10-04-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: how much orchestration is too much: early impressions - southden - 10-05-2026
Quote:What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A lot of
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: how much orchestration is too much: early impressions - quiethouse - 10-06-2026
Quote:[quote]What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: how much orchestration is too much: early impressions - labline - 10-07-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: how much orchestration is too much: early impressions - northgrid33 - 10-07-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher.
Re: how much orchestration is too much: early impressions - macronest - 10-07-2026
Ran a quick check on my box. On 4090 with vLLM, I saw basically the same behavior. The main thing that moved was latency, not quality.