+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Evals (https://www.datasculpting.ai/inference/forumdisplay.php?fid=66)
+--- Thread: benchmarks that map to real use (/showthread.php?tid=656)
benchmarks that map to real use - vramport - 10-02-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on A6000, I ended up with decent performance, but the setup only really made sense once I adjusted around VRAM headroom.
Nothing scientific here, just practical notes from a real setup.
Re: benchmarks that map to real use - tensorport - 10-04-2026
Quote:Did a bit of testing and thought I'd post the result here.
A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. I keep promising myself this is the last rebuild.
Re: benchmarks that map to real use - deltastation - 10-04-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.
Re: benchmarks that map to real use - signalpeak - 10-04-2026
What exact setup are you running? Are you testing this on A6000 or something else?
Re: benchmarks that map to real use - edgequill - 10-04-2026
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: benchmarks that map to real use - audiopoint18 - 10-05-2026
subscribed, curious where this lands.
Re: benchmarks that map to real use - cozywire - 10-05-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. I keep promising myself this is the last rebuild.
Re: benchmarks that map to real use - southspark - 10-05-2026
worth checking the driver first.
Re: benchmarks that map to real use - kernelgrid - 10-06-2026
subscribed, curious where this lands.
Re: benchmarks that map to real use - routeline23 - 10-06-2026
Appreciate the detail. The pattern around HBM constraints has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
If you post your exact settings, people here can probably narrow it down pretty fast.