synthetic data for support bots with SGLang

[routecraft]

Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about continuous batching.

Would rather learn from your scars than collect my own all at once.

I care more about stability than squeezing the last possible tokens/sec out of it.

[rocmplane]

Good point. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[audioforge]

A couple questions before I guess too hard. Is this local-only or are you routing anything externally?