human evals on small teams on mini PC

[fastbench]

Spent some time on this today and wanted to compare notes.

I've gone back and forth between SGLang and vLLM for DeepSeek. The interesting difference for me has been reranking.

Interested in tradeoffs more than fan loyalty.

[cacheline]

that fixed it for me too.

[promptpoint]

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[deltagrid]

same here on my side.

[macropoint]

Maybe, but I had the opposite result. I've seen people blame consumer GPUs when the real issue was much more mundane.

[modgarden]

Interesting, I hadn't tried that. I ended up in about the same place using Ollama. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[cacheline]

That matches what I was seeing. The pattern around embeddings has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[latentcraft]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.

[nanogarden]

worth checking the driver first.

[bytegarden73]

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.