DDR5
Ann Steffora Mutschler recent penned an article for Semiconductor Engineering that explores the role of new memory technology in the evolving data center. For example, DDR5 is expected to offer significant improvements over previous iterations of DRAM, including doubling the bandwidth and density over DDR4, as well as facilitating optimized channel efficiency. Nevertheless, DDR5 cannot stand alone as the industry struggles to meet the tsunami of data generated by an increasingly connected world.
“The amount of data being generated by sensors in a variety of connected devices across multiple markets is exploding and so is the demand for compute power to make sense of [it] all,” Mutschler explained. “Machine learning/AI/deep learning, separately and as extensions of markets filled with connected devices, are only adding to the need for additional types of memories.”
DRAM scaling slows
In addition to DDR5, the industry is eyeing GDDR5 (originally targeted at the graphics market), HBM2 and NVDIMM (persistent memory). This is because no single memory type is capable of satisfying the insatiable demand for ever more bandwidth and capacity, especially as Moore’s Law wanes and DRAM scaling slows.
“It’s been difficult for large memory manufacturers like Samsung, SK Hynix and Micron to get the densities higher and higher. Right now, 4- and 8-gigabit devices are shipping in mass production in high volumes,” Sarvagya Kochak, a senior product marketing manager at Rambus, told Semiconductor Engineering. “16 gigabit should be on the market in a year or two, but this poses a pretty significant challenge because if you look at it from a CPU perspective, CPU speeds haven’t gone up over the last decade. They have more or less leveled off because transistor speeds cannot be bumped up more than they are right now.”
To compensate, says Kochak, CPU vendors are adding more cores inside of processors, although this presents its own set of challenges.
“The problem with this approach is that within the CPU is an integrated memory controller, which interfaces with the DRAM DIMMs. With older CPUs, there would have been two memory controllers. Now there are four memory controllers. Soon there will be six memory controllers and the projection is that there may even be eight memory controllers per CPU,” he elaborated. “The reason why there are more memory controllers is because there needs to be more data to feed all of the cores, so if there are fewer memory controllers, the data can’t be fed from the DRAM to the cores. If the cores are hungry, there’s no point in having them. As the number of cores goes up, there is a requirement for more bandwidth from the DRAM into the system. If you look at the rate at which memory has been increasing, as well as the memory attach rate per CPU, it’s not a pretty picture.”
NVDIMMs (persistent memory)
As Kochcak previously explained in a detailed Semiconductor Engineering article, the above-mentioned DRAM challenges have prompted system and software architects to rethink standard memory-storage subsystems in an effort to extract all the performance they possibly can with currently available technologies. More specifically, the majority of application, OS and software stacks are (presently) designed with the underlying assumption that memory is volatile and the contents in DRAM can be lost. Complex mechanisms ensure little to no data is lost in the event of a power failure. Although this comes at the cost of performance, the industry has learned to accept the tradeoff.
With non-volatile DIMMs (NVDIMM) technology, it is possible to accelerate system performance by making memory persistent and eliminating the aforementioned tradeoff. As Mutschler observes, NVDIMMs operate as standard DRAM, while also offering the persistence of flash. A typical NVDIMM module incorporates DRAM, flash, control logic and an independent power source to retain critical in-memory data in the event of unexpected power loss, system crashes or planned shutdowns.
According to Kochak, companies such as Intel, Micron, Sony, Viking Technology, Samsung and Netlist are already shipping NVDIMMs. Moreover, the SNIA (the Storage Networking Industry Association) is working to promote the standard, while a separate group has formed to create a programming model for using persistent memories. This means application developers will ultimately become more of aware of how to fully exploit persistent memory.
“A lot of our software today is driven and developed with one key thing in mind—that DRAM is volatile, you can lose data that is in DRAM, so always design your system in a way that you can expect failures. To do that, there are a lot of things done such as checkpointing, logging and journaling,” he added. “And if you don’t have to do a lot of those operations, you can figure out ways to do things a lot faster and effectively with the same infrastructure and increase the application performance. Then you don’t need to do all of these other operations for dealing with volatile system memory because the system memory would be nonvolatile.”
Interested in learning more about memory in the changing data center? The full text of “Data centers turn to new memories,” by Ann Steffora Mutschler is available on Semiconductor Engineering here, while “Enabling higher system performance with NVDIMM-N” by Sarvagya Kochak can be read on Semiconductor Engineering here.
