Location-Aware Caching Mechanism for minimizing performance degradation induced by inefficient inter-chiplet data path

Jingyang Zheng ¹, Huajin Sun ², Yuning Zhan ¹, Chao Fu ^1,3, Zhiyuan Zhang ¹, Jun Han ^1
1 State Key Laboratory of Integrated Chips and Systems, Fudan University, Shanghai, 200433, China
² Shandong Bosuan Zhixin Information Technology Co., Ltd, Jinan, 250101, China
³ Shao-Chips Laboratory, Fudan University, Shaoxing, 312000, China

Abstract

Chiplet-based multicore CPU systems are increasingly recognized as a promising approach for performance scalability. However, this emerging paradigm poses new challenges for the last-level cache (LLC) organization. The high latency and limited bandwidth of the interposer, which connects multiple chiplets, significantly exacerbate inter-chiplet communication delays. Despite the growing importance of chiplet-based systems, research on optimizing LLC organization for these architectures remains limited. Existing strategies, originally developed for monolithic systems or multi-chip module GPUs, fall short in efficiently reducing data access latency.

This paper presents a Location-Aware Caching Mechanism (LACM) for the LLC, specifically designed for chiplet-based multicore systems. LACM replicates data across chiplets and enables the sharing of replicas within each chiplet, thereby reducing costly inter-chiplet communication. Furthermore, LACM incorporates a hardware predictor that enables LLC banks holding replicas to adaptively bypass remote home banks. This selective direct memory access mechanism optimizes both data access latency and cache utilization. We evaluate our work on a 4-chiplet, 64-core system, where LACM improves overall performance by 25.3% over the baseline and outperforms the state-of-the-art techniques Locality-Aware Replication, L1.5, and SAC by 20.3%, 8.8%, and 7.0%, respectively.

Keywords: Chiplet architecture, Multicore system, NUCA effect, Cache hierarchy

To read the full article, click here