long context that stayed useful for Llama

[emberquill]

Curious what people here are seeing.

I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling tail latency.

Would appreciate practical advice rather than theory.