prefill bottlenecks on longer prompts: what are you seeing?

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

prefill bottlenecks on longer prompts: what are you seeing?

amberpatch25 Offline

Offline

Member

Posts: 83
Threads: 6
Joined: Sep 2026
Reputation: 0

#11

10-14-2026, 09:53 PM

Good point. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

prefill bottlenecks on longer prompts: what are you seeing? - by oxidemixer - 10-04-2026, 11:41 PM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by vrampilot - 10-07-2026, 12:00 AM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by batchfield55 - 10-11-2026, 09:20 AM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by phaseshed - 10-08-2026, 12:02 AM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by cobalttinker - 10-11-2026, 12:47 AM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by southspark - 10-11-2026, 10:07 AM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by batchsize - 10-14-2026, 10:20 AM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by torchkite - 10-14-2026, 11:05 AM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by mangoquill - 10-14-2026, 12:42 PM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by plainline - 10-14-2026, 01:16 PM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by amberpatch25 - 10-14-2026, 09:53 PM

Re: prefill bottlenecks on longer prompts: what are you seeing? - by sparrowworks88 - 10-16-2026, 10:44 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)