+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Fine-Tuning / LoRA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=54)
+--- Thread: how much orchestration is too much: what are you seeing? (/showthread.php?tid=625)
Pages:
1
2
how much orchestration is too much: what are you seeing? - cudaline - 10-01-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of TensorRT-LLM and dual 3090 felt surprisingly usable once I tuned around embeddings.
Feel free to tear it apart if you see something dumb.
Re: how much orchestration is too much: what are you seeing? - bytegarden73 - 10-02-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: how much orchestration is too much: what are you seeing? - llm_wren - 10-02-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: how much orchestration is too much: what are you seeing? - southden - 10-02-2026
Quote:My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain path w
subscribed, curious where this lands.
Re: how much orchestration is too much: what are you seeing? - llm_wren - 10-03-2026
Quote:[quote]My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain
Interesting, I hadn't tried that. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: how much orchestration is too much: what are you seeing? - vectorport - 10-06-2026
Quote:[quote][quote]My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know th
I hit almost the same issue last night.
Re: how much orchestration is too much: what are you seeing? - northgrid33 - 10-06-2026
I'd start with the simpler stack.
Re: how much orchestration is too much: what are you seeing? - labline - 10-06-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: how much orchestration is too much: what are you seeing? - ggufvault - 10-07-2026
My first guess would be this: If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: how much orchestration is too much: what are you seeing? - llm_wren - 10-10-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. Once you know the plain path works, then add the faster path back in piece by piece.