disaggregated inference thoughts on M3 Max

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

disaggregated inference thoughts on M3 Max

cozyarch Offline

Junior Member

Posts: 28
Threads: 6
Joined: Aug 2026
Reputation: 0

08-22-2026, 04:51 PM

Has anyone else run into this?

My current setup is SGLang with 4090, and I keep running into prefix caching.

If you've tested something similar, what did you keep and what did you stop bothering with?

Find

« Next Oldest | Next Newest »

Messages In This Thread

disaggregated inference thoughts on M3 Max - by cozyarch - 08-22-2026, 04:51 PM

Re: disaggregated inference thoughts on M3 Max - by edgeriver - 08-22-2026, 05:41 PM

Re: disaggregated inference thoughts on M3 Max - by tensorfield88 - 08-23-2026, 05:47 PM

Re: disaggregated inference thoughts on M3 Max - by kernelmike - 08-24-2026, 02:37 AM

Re: disaggregated inference thoughts on M3 Max - by kernelmike - 09-02-2026, 02:09 PM

Re: disaggregated inference thoughts on M3 Max - by labline - 08-23-2026, 09:57 PM

Re: disaggregated inference thoughts on M3 Max - by cudapeak - 08-24-2026, 03:19 AM

Re: disaggregated inference thoughts on M3 Max - by cozygrid12 - 08-24-2026, 03:38 AM

Re: disaggregated inference thoughts on M3 Max - by kernelmike - 08-30-2026, 10:00 AM

Re: disaggregated inference thoughts on M3 Max - by kernelmike - 08-27-2026, 04:16 AM

Re: disaggregated inference thoughts on M3 Max - by rocmhouse90 - 08-27-2026, 05:12 AM

Re: disaggregated inference thoughts on M3 Max - by cudapeak - 08-30-2026, 05:39 AM

Re: disaggregated inference thoughts on M3 Max - by quantsupply - 08-30-2026, 10:46 AM

Re: disaggregated inference thoughts on M3 Max - by macropilot - 09-02-2026, 11:26 AM

Re: disaggregated inference thoughts on M3 Max - by tensorfield88 - 09-02-2026, 02:56 PM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com