how strict do you make schemas: worth it?

[amberplane]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on 7900 XTX, or squeeze more out of the stack and deal with the complexity.

What would you pick if you had to live with it for a few months? I keep promising myself this is the last rebuild.

[torchdesk]

I'd approach it in a boring order. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[dockbrook21]

A couple things I'd check first. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[ggufvault]

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.

[plaingarden18]

What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[orbitgarden]

Thanks, that helps. The pattern around NVFP4 has been pretty similar for me.

[mlxgarden]

Appreciate the detail. I ended up in about the same place using Ollama.

[cudaworks]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[routecrate]

Appreciate the detail. I ended up in about the same place using PyTorch.

[rocmmixer]

Appreciate the detail. I ended up in about the same place using PyTorch.

I hit almost the same issue last night.