10-07-2026, 09:57 AM
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Mostly CUDA, sometimes wrong, usually measured
|
batching changes after update for Llama
|
|
« Next Oldest | Next Newest »
|
Users browsing this thread: 1 Guest(s)