+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Evals (https://www.datasculpting.ai/inference/forumdisplay.php?fid=66)
+--- Thread: quality tests for local assistants (/showthread.php?tid=758)
quality tests for local assistants - kernelpeak - 10-04-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of Ollama and 4090 felt surprisingly usable once I tuned around embeddings.
Maybe useful as a starting point for newer folks.
Re: quality tests for local assistants - sparrowworks88 - 10-05-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
Re: quality tests for local assistants - pixelcrate82 - 10-05-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: quality tests for local assistants - amberrunner - 10-05-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: quality tests for local assistants - thetanest - 10-06-2026
Maybe, but I had the opposite result. I've seen people blame HBM constraints when the real issue was much more mundane.
Re: quality tests for local assistants - batchwire - 10-06-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at batch size.
Re: quality tests for local assistants - llm_wren - 10-06-2026
following this because I'm in the same boat.
Re: quality tests for local assistants - mangotinker73 - 10-06-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. Once you know the plain path works, then add the faster path back in piece by piece.