disaggregated inference thoughts with llama.cpp

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

disaggregated inference thoughts with llama.cpp

vramport Offline

Offline

Junior Member

Posts: 45
Threads: 6
Joined: Aug 2026
Reputation: 0

#2

09-11-2026, 08:47 AM

I tested this tonight. On mini PC with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.

mostly local

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

disaggregated inference thoughts with llama.cpp - by quietcraft - 09-11-2026, 04:26 AM

Re: disaggregated inference thoughts with llama.cpp - by vramport - 09-11-2026, 08:47 AM

Re: disaggregated inference thoughts with llama.cpp - by papermixer4 - 09-15-2026, 11:42 AM

Re: disaggregated inference thoughts with llama.cpp - by tensorfield88 - 09-11-2026, 09:48 AM

Re: disaggregated inference thoughts with llama.cpp - by rocmhouse90 - 09-12-2026, 09:51 AM

Re: disaggregated inference thoughts with llama.cpp - by kernelrunner - 09-13-2026, 10:29 AM

Re: disaggregated inference thoughts with llama.cpp - by vramport - 09-13-2026, 07:17 PM

Re: disaggregated inference thoughts with llama.cpp - by kernelrunner - 09-16-2026, 02:21 PM

Re: disaggregated inference thoughts with llama.cpp - by rocmpath - 09-15-2026, 07:31 AM

Re: disaggregated inference thoughts with llama.cpp - by stackarch - 09-16-2026, 11:36 AM

Re: disaggregated inference thoughts with llama.cpp - by localpatch - 09-15-2026, 12:21 PM

Re: disaggregated inference thoughts with llama.cpp - by kernelrunner - 09-15-2026, 04:55 PM

Re: disaggregated inference thoughts with llama.cpp - by sparrowkite53 - 09-16-2026, 05:39 AM

Re: disaggregated inference thoughts with llama.cpp - by rocmhouse90 - 09-16-2026, 10:17 AM

View a Printable Version

Forum Jump:

Users browsing this thread: