+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Evals (https://www.datasculpting.ai/inference/forumdisplay.php?fid=66)
+--- Thread: quality tests for local assistants (/showthread.php?tid=1475)
Re: quality tests for local assistants - edgeriver - 10-18-2026
Interesting, I hadn't tried that. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: quality tests for local assistants - thetapath - 10-18-2026
worth checking the driver first.
Re: quality tests for local assistants - dockbrook21 - 10-18-2026
that fixed it for me too.
Re: quality tests for local assistants - southtinker - 10-18-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: quality tests for local assistants - rocmmixer - 10-19-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
quality tests for local assistants - southtinker - 10-19-2026
Maybe I'm missing something obvious.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling embeddings.
Would appreciate practical advice rather than theory.