|
distill or just quantize
|
09-30-2026, 12:28 AM
Quote:I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did. If you post your exact settings, people here can probably narrow it down pretty fast.
team red, team patient
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
distill or just quantize - by quiettoken - 09-28-2026, 11:50 PM
Re: distill or just quantize - by quantshed - 09-30-2026, 12:00 AM
Re: distill or just quantize - by cozykite - 09-30-2026, 12:28 AM
Re: distill or just quantize - by oxidegrove40 - 09-30-2026, 01:08 AM
Re: distill or just quantize - by quietstack - 10-03-2026, 01:54 AM
Re: distill or just quantize - by quantshed - 10-03-2026, 02:37 AM
Re: distill or just quantize - by torchtrack24 - 10-03-2026, 02:53 AM
Re: distill or just quantize - by quantshed - 10-08-2026, 04:23 AM
Re: distill or just quantize - by batchscope72 - 10-06-2026, 03:17 AM
Re: distill or just quantize - by fusedriver - 10-06-2026, 03:36 PM
Re: distill or just quantize - by emberpeak - 10-09-2026, 05:13 PM
Re: distill or just quantize - by quantshed - 10-10-2026, 05:25 PM
Re: distill or just quantize - by quiettoken - 10-11-2026, 05:39 AM
Re: distill or just quantize - by northquill - 10-11-2026, 06:36 AM
|
