quality tests for local assistants

[edgeriver]

Interesting, I hadn't tried that. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[thetapath]

worth checking the driver first.

[dockbrook21]

that fixed it for me too.

[southtinker]

Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.

[rocmmixer]

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.

[southtinker]

Maybe I'm missing something obvious.

I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling embeddings.

Would appreciate practical advice rather than theory.