prefill/decode separation in practice

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

prefill/decode separation in practice

cacheline Offline

Senior Member

Posts: 341
Threads: 51
Joined: Sep 2026
Reputation: 0

#12

10-16-2026, 02:56 PM

Quote:Thanks, that helps. The pattern around reranking has been pretty similar for me. I got better results once I stopped chasing one metric and just looke

following this because I'm in the same boat.

measure twice, benchmark three times

Find

« Next Oldest | Next Newest »

Messages In This Thread

prefill/decode separation in practice - by thetaplane - 10-12-2026, 03:30 PM

Re: prefill/decode separation in practice - by kernelgrid - 10-13-2026, 03:46 AM

Re: prefill/decode separation in practice - by phaseport36 - 10-13-2026, 04:39 AM

Re: prefill/decode separation in practice - by cacheline - 10-13-2026, 05:04 AM

Re: prefill/decode separation in practice - by phasestream53 - 10-13-2026, 05:39 AM

Re: prefill/decode separation in practice - by batchsize - 10-13-2026, 06:12 AM

Re: prefill/decode separation in practice - by amberrunner - 10-13-2026, 07:44 AM

Re: prefill/decode separation in practice - by turboarch95 - 10-13-2026, 11:47 AM

Re: prefill/decode separation in practice - by ggufvault - 10-13-2026, 01:27 PM

Re: prefill/decode separation in practice - by kernelmike - 10-13-2026, 02:09 PM

Re: prefill/decode separation in practice - by turboarch95 - 10-18-2026, 05:58 AM

Re: prefill/decode separation in practice - by turboarch95 - 10-13-2026, 02:30 PM

Re: prefill/decode separation in practice - by cacheline - 10-16-2026, 02:56 PM

Re: prefill/decode separation in practice - by oxidemixer - 10-18-2026, 03:36 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com