10-13-2026, 02:56 AM
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
mostly local
Inference
›
Optimization & Performance
›
Context / KV Cache
speculative decoding actually worth it on dual 3090
speculative decoding actually worth it on dual 3090
|
speculative decoding actually worth it on dual 3090
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
speculative decoding actually worth it on dual 3090 - by cacheline - 10-11-2026, 12:04 PM
Re: speculative decoding actually worth it on dual 3090 - by quantnest - 10-11-2026, 04:28 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-12-2026, 05:18 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-12-2026, 06:40 PM
Re: speculative decoding actually worth it on dual 3090 - by nanopatch - 10-13-2026, 02:56 AM
Re: speculative decoding actually worth it on dual 3090 - by oxidestream - 10-15-2026, 12:39 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheplane98 - 10-13-2026, 03:09 PM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-13-2026, 03:58 PM
Re: speculative decoding actually worth it on dual 3090 - by dockbrook21 - 10-15-2026, 04:35 AM
Re: speculative decoding actually worth it on dual 3090 - by cacheline - 10-15-2026, 02:57 PM
|
Users browsing this thread: 1 Guest(s)