Last month, Semiconductor Engineering’s Kevin Fogarty wrote an article that explores how major industry players are pushing the limits of DRAM. As Fogarty observes, the access bandwidth of DRAM-based computer memory has improved by a factor of 20x over the past two decades – with capacity increasing 128x during the same period. In contrast, latency has only improved 1.3x.
“Modern computers, especially data center servers that skew heavily toward in-memory databases, data-intensive analytics and increasingly toward machine-learning and deep-neural-network training functions, depend on large amounts of high-speed, high-capacity memory to keep the wheels turning,” he explains. “Despite years of effort by researchers looking for better, faster alternatives, DRAM remains a near-universal choice when performance is the priority.”
Indeed, DDR5, the next-gen DRAM specification from JEDEC, will have twice the density and twice the bandwidth of DDR4. Combined with accelerators, DDR5 is expected to make a major difference in the data center for performance-intensive, time-sensitive FinTech applications and other high-end analytic, HPC and supercomputing applications.
“There is clearly a need for more memory bandwidth and more memory capacity, but DDR5 won’t be enough by itself and it’s not clear which of several other approaches may take off,” Steven Woo, VP of Systems and Solutions, Office of the CTO, Rambus, told Semiconductor Engineering during a recent interview with the publication. “We’re already seeing a lot of processing cycles moving away from traditional x86 processes – more mining of cryptocurrencies and training neural networks, moving toward GPU and specialized silicon, or even morphing of architectures to shift some of the processing closer to the storage in the data center or as edge or fog computing.”
In addition to highlighting the upcoming DDR5 specification, Fogarty also discusses potential methods of overcoming DRAM thermal limitations in the data center, such as cryogenic, or cold computing.
“If you could reliably pull out the heat, you could pack memory, processors and graphics co-processors much more tightly,” he elaborates. “[This would] save space that can be used for more servers and improving performance by reducing lag between memory and all the other components of the system.”
In addition, writes Fogarty, when CMOS is cold enough, data leaks from a CMOS chip can stop completely, with the silicon becoming almost non-volatile.
“Performance increases to the point that memory could catch up to the speed of processors, eliminating one of the most stubborn bottlenecks in the IC industry,” he continues. “At very cold temperatures, between 4K and 7K, wires effectively superconduct, allowing the chip to communicate over long distances using very little energy.”
As Fogarty reports, Rambus has been working with Microsoft since 2015 on memory for quantum computing as part of Microsoft’s effort to build a topological quantum computer. Since the quantum processor has to operate at cryogenic temperatures – below -292° F/-180°C or 93.15K — so did the DRAM Rambus was testing for the project.
According to Craig Hampel, chief scientist for Rambus’ Memory and Interface Division, liquid nitrogen is cheap – tens of cents per gallon – and the cost curve doesn’t get really steep until one approaches supercooling, around 4 Kelvin.
“Down to about 50 Kelvin, it’s not that expensive,” he says. “Taking out the heat [also] lets you compress the size of the server rack as much as 70%, which means the density per cubic foot of data center improves. That makes them easier to maintain and easier to place in areas you couldn’t reach before.”
Interested in learning more? The full text of “Pushing DRAM’s Limits” by Kevin Fogarty is available on Semiconductor Engineering here and “The Case for Cold DRAM in the Data Center” can be read on the Rambus blog here.
