Hacker News

Processing in Memory: DRAM Is About to Do Math

29 points by bhouston 3 days ago | 7 comments

honr 42 minutes ago

Compute In Memory has been most of what electronics research (more accurately, the parts I am interacting with) talking about this year. There are a few trends and I am curious to see which ones survive in the market. RAM designers are hot in the market and job openings at the moment.

paretolaw 14 minutes ago

So, long samsung and short nvidia

RetroTechie 2 days ago

AI focussed - as so much these days. Some corp. please put general-purpose cores in that RAM?

ghaff 2 hours ago

I looked at this for a client close to 20 years ago. The problem is that you need to get software support for it. Arguably easier these days with open source but still doesn't magically happen.

MadnessASAP 2 hours ago

AI inference is basically 1 operation iterated over every single model weight. That's what's makes computer in memory attractive and feasible.

General purpose workloads would require much more silicon real estate and are not (typically) nearly as bandwidth hungry. Just like not all CPU workloads benefit from being adapted to a GPU, even fewer will benefit from being adapted to processing in RAM.

Lerc 20 minutes ago

AI is not intrinsically matmul. Current models are a framing of the problem as matmul to enable the bulk operations necessary. There are many calculations that can (and have been) reframed into matrix form to achieve similar results. More than once I have seen a matmul version of something that I thought couldn't have been representable that way. I'm not sure if the greater benefit is the performance or the ability to conceptually manage the problem in you head.

Not all algorithms can be converted of course, but a lot of that is because there are some inherently unparallelizable problems.

peter_d_sherman 2 days ago

I like the idea of a CPU/GPU/TPU on RAM (look at those gorgeous bandwidth numbers!) -- but I also wonder what problems it may potentially bring in the future... I think it would be a good idea that if RAM is to implement processors / PIM on it in the future, that each RAM stick should have an external DIP "safety" switch on the side which switches off such processing. If this switch is flipped by the user, then the stick simply falls back to operating as normal "dumb" RAM, driven by external CPU control. Also, it would be a good idea to have debug ports, debug proxying, single step capability, etc., in these futuristic RAM sticks, as well...

"SIMDRAM: A Framework for Bit-Serial SIMD Processing Using DRAM Extended Abstract" (2020):

https://arxiv.org/html/2012.11890v1

>"We leverage the SIMDRAM framework to accelerate seven application kernels from machine learning, databases, and image processing (VGG-13 [63], VGG-16 [63], LeNET [38], kNN [42], TPC-H [66], BitWeaving [45], Brightness [28]). SIMDRAM provides up to 2.5× speedup for the kernels compared to Ambit [58]. Compared to a CPU and a high-end GPU, SIMDRAM is 257× and 31× more energy efficient, while providing 93 × and 6× higher throughput, respectively."