sandboxing agent actions: worth it?

[minthall]

I'm trying to sanity-check my setup.

My current setup is PyTorch with 7900 XTX, and I keep running into consumer GPUs.

Would appreciate practical advice rather than theory.

[llmtrack]

Appreciate the detail. I ended up in about the same place using LM Studio. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[macrostack69]

Thanks, that helps. The pattern around HBM constraints has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer. I keep promising myself this is the last rebuild.

[quietarch]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.

[kernelmike]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.

I'd approach it in a boring order. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[torchtrack24]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.

Good point. I don't think your conclusion is far off.

[torchdesk]

[quote]I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.

[/qu

My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.

[batchfield55]

Need a little more detail here. What model, what runtime, and roughly what context size?

If you post your exact settings, people here can probably narrow it down pretty fast.

[relayforge]

Interesting, I hadn't tried that. That matches what I saw on GB200.

[sparrowcraft]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4.