structured output failures

[vramport]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on 4090, or squeeze more out of the stack and deal with the complexity.

Interested in tradeoffs more than fan loyalty.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

[benchplease]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.

[cobaltlab]

What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks.