10-04-2026, 02:21 AM
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Inference
›
Drivers / Frameworks / Toolkits
›
NVIDIA / CUDA
batching changes after update with llama.cpp
batching changes after update with llama.cpp
|
batching changes after update with llama.cpp
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
batching changes after update with llama.cpp - by dockember - 10-02-2026, 12:41 AM
Re: batching changes after update with llama.cpp - by thetapath - 10-02-2026, 05:09 AM
Re: batching changes after update with llama.cpp - by audioriver - 10-02-2026, 06:14 AM
Re: batching changes after update with llama.cpp - by latentgarden - 10-02-2026, 02:54 PM
Re: batching changes after update with llama.cpp - by audiopath66 - 10-02-2026, 05:17 PM
Re: batching changes after update with llama.cpp - by latentgarden - 10-03-2026, 01:36 AM
Re: batching changes after update with llama.cpp - by orbitbench6 - 10-04-2026, 02:21 AM
|
Users browsing this thread: 1 Guest(s)