Getting 50 GB/S Back from the Apple Neural Engine
44 points by eiln 3 days ago | 10 comments
bee_rider 36 minutes ago
Nice investigation.
replyIt is always surprising to me when a nice round number like 1MiB results in the “bad performance” configuration (although it happens).
Are you sure erratum is the right word in this context? I usually see it used to describe the notice that a document has an error in it.
thenewwazoo 24 minutes ago
"Apparently the memory controller's throughput has a dominant harmonic with wavelength 2048 in tensor-dimension space."
replyThat got a laugh out of me.
Neywiny 2 hours ago
Just checking here- this systemverilog is a hypothetical telling of what you think is going on? Or do you have the actual source of the RTL?
replyeiln 3 days ago
RTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s.
reply
For example, if I browse HN, I open the linked article in a new tab instead of in the same one where the comments are. And I close the tab as soon as I’m done looking at it. Whether that be a couple of seconds because of annoying dark patterns, or a long time thanks to being actually readable and interesting. From that tab I might navigate to another page on the same site if the site seems good. If they then pull horse crap with the browser history in that tab I will most likely close the tab and not read anything more on that site. If I repeatedly get annoyed by a domain from having gone to it and found dark patterns I will eventually remember that the whole site is bad and stop clicking any links to it either. But mostly I don’t have to remember so many sites to actively avoid. Closing a tab is cheap when all of the history is from the same site that tab was opened for in the first place.
It’s unfortunate and annoying that they get to play tricks with dark patterns still, and I wish nobody did that / could do that in the first place. In general I mean.
I usually don’t comment on it even if I do come across that. As I know many consider it off topic. But I still appreciate when others, like your comment here, point those things out and I then don’t click on to the linked article in the first place when I see comments about that unless other comments are indicating that the content itself is particularly interesting.