+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Fine-Tuning / LoRA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=54)
+--- Thread: how much orchestration is too much (/showthread.php?tid=1310)
Pages:
1
2
Re: how much orchestration is too much - thetanest - 10-19-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: how much orchestration is too much - llm_wren - 10-19-2026
Quote:My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at HBM constraints.
Did a small comparison this afternoon. On 7900 XTX with LM Studio, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: how much orchestration is too much - deepseeked - 10-19-2026
That matches what I was seeing. That matches what I saw on 4090.
Re: how much orchestration is too much - oxidegrove40 - 10-19-2026
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: how much orchestration is too much - llm_wren - 10-19-2026
I think the bottleneck is somewhere else. I get why that seems right, but on Threadripper box the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: how much orchestration is too much - labline - 10-19-2026
That matches what I was seeing. That matches what I saw on dual 3090. The biggest change for me was simplifying the stack instead of adding one more optimization layer.