+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: throughput per watt reality check for Mistral (/showthread.php?tid=478)
throughput per watt reality check for Mistral - batchscope72 - 09-25-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between Ollama and llama.cpp for Llama. The interesting difference for me has been tail latency.
What would you pick if you had to live with it for a few months?