+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: CPUs / NPUs / Accelerators (https://www.datasculpting.ai/inference/forumdisplay.php?fid=30)
+--- Thread: do small edge boxes make sense: early impressions (/showthread.php?tid=1214)
Pages:
1
2
Re: do small edge boxes make sense: early impressions - latentscope35 - 10-19-2026
Thanks, that helps. That matches what I saw on 4090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: do small edge boxes make sense: early impressions - quantsupply - 10-19-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4.
Re: do small edge boxes make sense: early impressions - routecraft - 10-19-2026
Quote:I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4.
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.