09-27-2026, 01:15 AM
Has anyone else run into this?
My current setup is llama.cpp with 4090, and I keep running into consumer GPUs.
If you've tested something similar, what did you keep and what did you stop bothering with?
I care more about stability than squeezing the last possible tokens/sec out of it.
The part that surprised me was how much the boring details mattered more than the model choice.
My current setup is llama.cpp with 4090, and I keep running into consumer GPUs.
If you've tested something similar, what did you keep and what did you stop bothering with?
I care more about stability than squeezing the last possible tokens/sec out of it.
The part that surprised me was how much the boring details mattered more than the model choice.
context windows are not a product strategy