|
long context that stayed useful
|
10-01-2026, 09:24 PM
Quote:[quote]My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tail latency. Once you Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
context windows are not a product strategy
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
long context that stayed useful - by ggufvault - 09-25-2026, 08:47 PM
Re: long context that stayed useful - by phasespark - 09-25-2026, 10:35 PM
Re: long context that stayed useful - by ggufvault - 09-26-2026, 07:16 AM
Re: long context that stayed useful - by ggufvault - 09-29-2026, 09:07 AM
Re: long context that stayed useful - by amberkite - 09-29-2026, 07:24 AM
Re: long context that stayed useful - by edgequill - 09-30-2026, 09:08 AM
Re: long context that stayed useful - by flintriver50 - 10-01-2026, 09:08 PM
Re: long context that stayed useful - by tinycontext - 10-01-2026, 09:24 PM
Re: long context that stayed useful - by flintriver50 - 10-02-2026, 09:43 AM
Re: long context that stayed useful - by quiethouse - 10-02-2026, 10:37 AM
Re: long context that stayed useful - by visionforge - 10-02-2026, 11:20 AM
|
