Corsair: An In-memory Computing Chiplet Architecture for Inference-time Compute Acceleration
By Satyam Srivastava; Akhil Arunkumar; Nithesh Kurella; Amrit Panda; Gaurav Jain; Purushotham Kamath
d-Matrix Corporation
Abstract:
Advances in Generative AI (GenAI) have reinvigorated research into novel computing architectures such as Transformer. Transformer, characterized by low arithmetic intensity during most of the inference time, has become the cornerstone of GenAI underlying Large Language (LLM) and Reasoning Models (RM). Numerous solutions to the intense memory bandwidth problem have been proposed. Corsair is an architecture that targets this need using chiplet design, digital in-memory computing-based matrix engine, efficient die-to-die interconnects, block floating point numerics, and large high-bandwidth on-chip memories. We describe the Corsair chiplet, scaling approaches to compose larger systems, and outline the software stack. We formulate the inference-time requirements of LLM and RM computation, memory bandwidth, memory capacity, and interconnect efficiency for scaling. We also show how Corsair design perfectly fits these workloads. We present benchmark results from Corsair silicon that correlate strongly with the design and preview an estimate of workload-level improvements expected with Corsair.
To read the full article, click here
Related Chiplet
- 12nm EURYTION RFK1 - UCIe SP based Ka-Ku Band Chiplet Transceiver
- Interconnect Chiplet
- Bridglets
- Automotive AI Accelerator
- Direct Chiplet Interface
Related Technical Papers
- Gemini: Mapping and Architecture Co-exploration for Large-scale DNN Chiplet Accelerators
- AIG-CIM: A Scalable Chiplet Module with Tri-Gear Heterogeneous Compute-in-Memory for Diffusion Acceleration
- A Heterogeneous Chiplet Architecture for Accelerating End-to-End Transformer Models
- Universal Chiplet Interconnect Express: An Open Industry Standard for Memory and Storage Applications
Latest Technical Papers
- Corsair: An In-memory Computing Chiplet Architecture for Inference-time Compute Acceleration
- Thermal Implications of Non-Uniform Power in BSPDN-Enabled 2.5D/3D Chiplet-based Systems-in-Package using Nanosheet Technology
- Chiplets Are The New Baseline for AI Inference Chips
- Hybrid Bonding With Polymeric Interlayer Dielectric Layers Patterned by Nanoimprint Lithography
- Advanced Optical Integration Processes for Photonic-Integrated Circuit Packaging