Getting 50 GB/S Back from the Apple Neural Engine

(eiln.github.io)

32 points | by eiln 2 days ago

5 comments

  • VladVladikoff 28 minutes ago
    This website hijacked my back button during a simple page load. You should fix that, it’s not an acceptable way to behave.
    • switchbak 6 minutes ago
      I’m shocked that’s still possible, shouldn’t we be able to fix that at the browser level?
      • busymom0 3 minutes ago
        Just like how browsers disable or mute auto playing videos, they should be able to prevent back button hijacking until user has clicking a link or button.
  • bee_rider 5 minutes ago
    Nice investigation.

    It is always surprising to me when a nice round number like 1MiB results in the “bad performance” configuration (although it happens).

    Are you sure erratum is the right word in this context? I usually see it used to describe the notice that a document has an error in it.

  • Neywiny 52 minutes ago
    Just checking here- this systemverilog is a hypothetical telling of what you think is going on? Or do you have the actual source of the RTL?
  • eiln 2 days ago
    RTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s.
  • RantyDave 7 minutes ago
    Ummm, wow. That's really bad.