8B vs 14B sweet spot with TensorRT-LLM

[southspark]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around MLX and M3 Max.

Simple and boring is a valid answer.

[fastfield4]

A couple questions before I guess too hard. What model, what runtime, and roughly what context size?

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.