tokens/sec tuning ideas: worth it?

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

tokens/sec tuning ideas: worth it?

macronest Offline

Offline

Junior Member

Posts: 38
Threads: 7
Joined: Oct 2026
Reputation: 0

#1

09-29-2026, 01:11 AM

Thought I'd post a small recipe because a lot of threads ask similar things.

Just sharing a setup that ended up working reasonably well: Mixtral, SGLang, A6000, with a lot of attention on consumer GPUs.

Feel free to tear it apart if you see something dumb.

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

tokens/sec tuning ideas: worth it? - by macronest - 09-29-2026, 01:11 AM

Re: tokens/sec tuning ideas: worth it? - by northhall88 - 09-30-2026, 01:40 AM

Re: tokens/sec tuning ideas: worth it? - by marbletrack - 10-01-2026, 02:30 AM

Re: tokens/sec tuning ideas: worth it? - by cozygrid12 - 10-01-2026, 03:38 AM

Re: tokens/sec tuning ideas: worth it? - by vectorhouse71 - 10-02-2026, 04:08 PM

Re: tokens/sec tuning ideas: worth it? - by northgarden37 - 10-02-2026, 08:48 PM

Re: tokens/sec tuning ideas: worth it? - by northgrid33 - 10-03-2026, 09:26 PM

Re: tokens/sec tuning ideas: worth it? - by cachelab - 10-03-2026, 10:31 PM

Re: tokens/sec tuning ideas: worth it? - by macrotinker - 10-04-2026, 10:34 AM

Re: tokens/sec tuning ideas: worth it? - by sparrowghost - 10-04-2026, 06:56 PM

Re: tokens/sec tuning ideas: worth it? - by fastfield4 - 10-04-2026, 08:37 PM

Re: tokens/sec tuning ideas: worth it? - by cudaworks - 10-07-2026, 08:59 PM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)