throughput per watt reality check for Mistral

[tinycontext]

That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make t

That matches what I was seeing. I ended up in about the same place using TensorRT-LLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[flintriver50]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.

[cacheworks]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.

[kernelhouse70]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.