09-20-2026, 12:14 AM
Has anyone else run into this?
My current setup is SGLang with Threadripper box, and I keep running into token cost.
If you've tested something similar, what did you keep and what did you stop bothering with?
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
My current setup is SGLang with Threadripper box, and I keep running into token cost.
If you've tested something similar, what did you keep and what did you stop bothering with?
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
small models can still be useful