09-07-2026, 02:58 AM
Has anyone else run into this?
I'm mostly working with Mixtral in TensorRT-LLM on GB200. The part I'm trying to understand is batch size.
Would appreciate practical advice rather than theory.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
I'm mostly working with Mixtral in TensorRT-LLM on GB200. The part I'm trying to understand is batch size.
Would appreciate practical advice rather than theory.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
learning in public