+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Agents (https://www.datasculpting.ai/inference/forumdisplay.php?fid=44)
+--- Thread: student models for local use (/showthread.php?tid=221)
Pages:
1
2
Re: student models for local use - cacheline - 09-14-2026
subscribed, curious where this lands.
Re: student models for local use - llmtrack26 - 09-14-2026
I tested this tonight. On mini PC with SGLang, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: student models for local use - rocmmixer - 09-14-2026
I hit almost the same issue last night.
Re: student models for local use - amberpatch25 - 09-17-2026
What exact setup are you running? Are you testing this on dual 3090 or something else?
Re: student models for local use - quantshed - 09-17-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: student models for local use - amberpatch25 - 09-17-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.