do small edge boxes make sense: early impressions

[oxiderunner]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on dual 3090, or squeeze more out of the stack and deal with the complexity.

Trying to avoid rebuilding the whole thing twice.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.

[bytelab]

Spent some time on this today and wanted to compare notes.

I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[oxidestream]

Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher.

[fastscope]

Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?

[fusedriver]

That matches what I was seeing. The pattern around HBM constraints has been pretty similar for me.

[oxideworks59]

That matches what I was seeing. The pattern around HBM constraints has been pretty similar for me.

worth checking the driver first.

[dockworks23]

I hit almost the same issue last night.

[cacheplane98]

I tested this tonight. On A6000 with SGLang, I saw basically the same behavior.

[tensorkite]

I tested this tonight. On A6000 with SGLang, I saw basically the same behavior.

A couple questions before I guess too hard. Is this local-only or are you routing anything externally?

[rocmriley]

Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.