dataset format recommendations

[plainworks]

[quote][quote]Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency

I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.