+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: CPUs / NPUs / Accelerators (https://www.datasculpting.ai/inference/forumdisplay.php?fid=30)
+--- Thread: do small edge boxes make sense: early impressions (/showthread.php?tid=1214)
Pages:
1
2
do small edge boxes make sense: early impressions - oxiderunner - 10-14-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on dual 3090, or squeeze more out of the stack and deal with the complexity.
Trying to avoid rebuilding the whole thing twice.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Re: do small edge boxes make sense: early impressions - bytelab - 10-16-2026
Quote:Spent some time on this today and wanted to compare notes.
I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: do small edge boxes make sense: early impressions - oxidestream - 10-16-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher.
Re: do small edge boxes make sense: early impressions - fastscope - 10-17-2026
Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?
Re: do small edge boxes make sense: early impressions - fusedriver - 10-17-2026
That matches what I was seeing. The pattern around HBM constraints has been pretty similar for me.
Re: do small edge boxes make sense: early impressions - oxideworks59 - 10-17-2026
Quote:That matches what I was seeing. The pattern around HBM constraints has been pretty similar for me.
worth checking the driver first.
Re: do small edge boxes make sense: early impressions - dockworks23 - 10-18-2026
I hit almost the same issue last night.
Re: do small edge boxes make sense: early impressions - cacheplane98 - 10-18-2026
I tested this tonight. On A6000 with SGLang, I saw basically the same behavior.
Re: do small edge boxes make sense: early impressions - tensorkite - 10-18-2026
Quote:I tested this tonight. On A6000 with SGLang, I saw basically the same behavior.
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: do small edge boxes make sense: early impressions - rocmriley - 10-18-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.