on-device inference worth the effort: what are you seeing?

[sparrowshed]

I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.