09-28-2026, 12:41 AM
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: MLX on dual 3090, keeping continuous batching in mind from the start.
Feel free to tear it apart if you see something dumb.
The part that surprised me was how much the boring details mattered more than the model choice.
Maybe useful as a starting point: MLX on dual 3090, keeping continuous batching in mind from the start.
Feel free to tear it apart if you see something dumb.
The part that surprised me was how much the boring details mattered more than the model choice.