|
QLoRA resource reality check with SGLang
|
09-23-2026, 02:09 AM
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
I'm happy to test one more variant if you want a second data point.
throughput matters, but tail latency matters too
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
QLoRA resource reality check with SGLang - by deepseeked - 09-22-2026, 01:34 AM
Re: QLoRA resource reality check with SGLang - by batchsize - 09-23-2026, 02:09 AM
Re: QLoRA resource reality check with SGLang - by batchsize - 09-23-2026, 03:47 PM
Re: QLoRA resource reality check with SGLang - by batchpeak - 09-23-2026, 02:56 PM
Re: QLoRA resource reality check with SGLang - by routearch - 09-24-2026, 08:22 AM
Re: QLoRA resource reality check with SGLang - by benchplease - 09-24-2026, 03:55 AM
Re: QLoRA resource reality check with SGLang - by tensorfield88 - 09-24-2026, 12:44 PM
|
