Workload-informed System-aware approaches to Exploiting Heterogeneous Memory Systems

The memory subsystem is one critical component for large-scale parallel systems for cost and performance. As the memory wall and memory bandwidth wall challenges persist, active development in new memory devices, such as non-volatile memories and high-bandwidth memories, brings heterogeneous memory systems (HMS) as a promising solution for implementing large-scale memory systems within cost, area, and power limitations. Typical HMS consists of a small-capacity high-performance tier and a large-capacity low-performance tier. On high-end large-scale systems, such HMS systems have evolved from HBM-DRAM systems to later byte-addressable NVM-DRAM systems to the recent CXL-interconnected disaggregated memory tier(s). While hardware-managed cache can significantly ease programmability and adoption, extensive works have shown their limitations when workloads have complex memory access patterns and large memory footprints, such as graph processing workloads. Thus, software-based data placement on such systems plays a vital role in performance optimization. In this talk, approaches that combine workload knowledge with system-level information will be presented for several different types of HMS. Software-based approaches that combine workload knowledge and system-level states, such as coarse-grained data placement for dense data structures and runtime solutions for adaptive granularity data placement optimization, will be presented.