speculative decoding actually worth it on dual 3090
|
speculative decoding actually worth it on dual 3090
|
10-13-2026, 03:58 PM
Quote:A couple questions before I guess too hard. Are you testing this on 5090 or something else? Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
measure twice, benchmark three times
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
speculative decoding actually worth it on dual 3090 - by cacheline - 10-11-2026, 12:04 PM
Re: speculative decoding actually worth it on dual 3090 - by quantnest - 10-11-2026, 04:28 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-12-2026, 05:18 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-12-2026, 06:40 PM
Re: speculative decoding actually worth it on dual 3090 - by nanopatch - 10-13-2026, 02:56 AM
Re: speculative decoding actually worth it on dual 3090 - by oxidestream - 10-15-2026, 12:39 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheplane98 - 10-13-2026, 03:09 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-13-2026, 03:58 PM
Re: speculative decoding actually worth it on dual 3090 - by dockbrook21 - 10-15-2026, 04:35 AM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-15-2026, 02:57 PM
|
