Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
odd little inference appliances with MLX
#6
Quote:I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at VRAM headroom.

I don't think that's the whole story. I've seen people blame structured output when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Reply


Messages In This Thread
Re: odd little inference appliances with MLX - by bytepilot - 10-12-2026, 10:57 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)