10-18-2026, 12:57 PM
I tested this tonight. On 5090 with llama.cpp, I saw basically the same behavior. The main thing that moved was latency, not quality.
2x3090 • llama.cpp • too much VRAM is still not enough
|
prefill/decode separation in practice on B200
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
prefill/decode separation in practice on B200 - by cacheline - 10-15-2026, 12:34 AM
Re: prefill/decode separation in practice on B200 - by thetashed24 - 10-15-2026, 12:37 PM
Re: prefill/decode separation in practice on B200 - by signalcraft14 - 10-18-2026, 12:42 PM
Re: prefill/decode separation in practice on B200 - by vrambandit - 10-18-2026, 12:57 PM
Re: prefill/decode separation in practice on B200 - by amberrunner - 10-18-2026, 02:28 PM
Re: prefill/decode separation in practice on B200 - by batchsize - 10-18-2026, 03:34 PM
Re: prefill/decode separation in practice on B200 - by batchember - 10-18-2026, 04:10 PM
|
Users browsing this thread: 1 Guest(s)