Inference › Inference Engines › Other Engines v
tokens/sec tuning ideas

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
tokens/sec tuning ideas
kernelmike Offline
Member
***
Posts: 184
Threads: 23
Joined: Sep 2026
Reputation: 0
#1
10-18-2026, 04:52 PM
This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like structured output decides everything downstream.

Wondering if this is just where the whole space is heading now.
Mostly CUDA, sometimes wrong, usually measured
Find
Reply


Messages In This Thread
tokens/sec tuning ideas - by kernelmike - 10-18-2026, 04:52 PM
Re: tokens/sec tuning ideas - by northgarden37 - 10-18-2026, 06:00 PM
Re: tokens/sec tuning ideas - by sysstation69 - 10-18-2026, 06:05 PM
Re: tokens/sec tuning ideas - by northquill - 10-18-2026, 09:46 PM
Re: tokens/sec tuning ideas - by cacheworks - 10-19-2026, 12:00 AM
Re: tokens/sec tuning ideas - by rocmplane - 10-19-2026, 01:55 AM
Re: tokens/sec tuning ideas - by cachebench - 10-19-2026, 02:16 AM
Re: tokens/sec tuning ideas - by amberpatch25 - 10-19-2026, 03:23 AM
Re: tokens/sec tuning ideas - by vramstream - 10-19-2026, 04:24 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)