+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Recipes & Configurations (https://www.datasculpting.ai/inference/forumdisplay.php?fid=52)
+---- Thread: routing between local and hosted: what are you seeing? (/showthread.php?tid=161)
routing between local and hosted: what are you seeing? - oxidegrove40 - 09-07-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around PyTorch and 7900 XTX.
I'm especially interested in mistakes worth avoiding. The logs were somehow calmer than I was.
Re: routing between local and hosted: what are you seeing? - deepseeked - 09-07-2026
I tested this tonight. On mini PC with TensorRT-LLM, I saw basically the same behavior.
Re: routing between local and hosted: what are you seeing? - tensorworks - 09-08-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: routing between local and hosted: what are you seeing? - visionforge - 09-09-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: routing between local and hosted: what are you seeing? - mintstation99 - 09-09-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: routing between local and hosted: what are you seeing? - signalcraft14 - 09-09-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. Once you know the plain path works, then add the faster path back in piece by piece.
Re: routing between local and hosted: what are you seeing? - batchsize - 09-10-2026
Quote:A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. Once
I hit almost the same issue last night.
I'm happy to test one more variant if you want a second data point.
Re: routing between local and hosted: what are you seeing? - mangoquill - 09-10-2026
same here on my side.
Re: routing between local and hosted: what are you seeing? - localpatch - 09-10-2026
What exact setup are you running? Are you testing this on mini PC or something else?