memory layers vs plain long context: setup advice

[torchkite]

Appreciate the detail. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumption can ma

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.