+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: tokens/sec tuning ideas (/showthread.php?tid=374)
tokens/sec tuning ideas - sparrowworks88 - 09-20-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: SGLang on mini PC, keeping INT4 in mind from the start.
Maybe useful as a starting point for newer folks.
Re: tokens/sec tuning ideas - plaingarden18 - 09-20-2026
What exact setup are you running? What model, what runtime, and roughly what context size?