+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Security, Privacy & Safety (https://www.datasculpting.ai/inference/forumdisplay.php?fid=23)
+--- Forum: Sandboxing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=78)
+--- Thread: best sandbox for trying risky tools (/showthread.php?tid=709)
Pages:
1
2
best sandbox for trying risky tools - vramghost - 10-03-2026
This keeps coming up in conversation so I figured I'd start a thread.
It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with INT4.
Maybe obvious to some of you, but it clicked for me this week.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
Re: best sandbox for trying risky tools - phasenode - 10-04-2026
Tried to reproduce it just now. On B200 with MLX, I saw basically the same behavior.
Re: best sandbox for trying risky tools - amberkite - 10-04-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: best sandbox for trying risky tools - edgekite - 10-05-2026
I think the bottleneck is somewhere else. I get why that seems right, but on B200 the bottleneck felt elsewhere.
Re: best sandbox for trying risky tools - cozyarch - 10-05-2026
Thanks, that helps. I ended up in about the same place using MLX. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: best sandbox for trying risky tools - localtinker - 10-05-2026
Quote:Thanks, that helps. I ended up in about the same place using MLX. Worth rechecking your defaults though, because one bad assumption can make the rest
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: best sandbox for trying risky tools - rocmpath - 10-05-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. I keep promising myself this is the last rebuild.
Re: best sandbox for trying risky tools - mangohouse - 10-08-2026
worth checking the driver first.
Re: best sandbox for trying risky tools - audiodesk - 10-08-2026
Quote:worth checking the driver first.
I'd start with the simpler stack.
I'm happy to test one more variant if you want a second data point.
Re: best sandbox for trying risky tools - southden - 10-11-2026
that fixed it for me too.