Unleashing AI Potential Through Advanced Chiplet Architectures

By Tony Chan Carusone, CTO, Alphawave Semi

December 11, 2024

The rapid proliferation of machine-generated data is driving unprecedented demand for scalable AI infrastructure, placing extreme pressure on compute and connectivity within data centers. As the power requirements and carbon footprint of AI workloads rise, there is a critical need for efficient, high-performance hardware solutions to meet growing demands. Traditional monolithic ICs will not scale.  Thus, chiplet architectures are playing a critical role in scaling AI.

Combining chiplets via low-latency and high-bandwidth connections across modular, custom components facilitates performance growth beyond the reticle limit. Connectivity standards such as UCIe enable seamless inter-die communication. Chiplets also support AI scale-up and scale-out.  Even distributed AI across multiple sites benefits from chiplet architectures.

Harnessing the chiplet ecosystem to design flexible, interoperable compute and connectivity within a single package optimized for workloads is the only way to sustainably scale AI.

Data is proliferating

Machine generated data is proliferating like never before. The size of the data sphere will reach 181 billion terabytes (181 ZB) next year and the need to scale AI has accelerated new and upgraded data center infrastructure.  However, processing isn’t the only limit – improvements in connectivity will also be critical in scaling AI.

A decade ago, data was primarily generated by people interacting with technology, and its growth was linear. With autonomous sensor and video data, financial data and yet more data produced by analyzing other data, the growth became exponential.

This is driving a focus on AI to parse all this data. Compute infrastructure is being pushed to the absolute limit imposed by the performance of a single full reticle-sized monolithic die. Hardware cost is a significant concern since deploying AI at scale may require, for example, 8 GPUs per server across 20,000 servers at a cost of around $4 billion. Energy is also a limiting factor representing millions of dollars in operational costs.  Moreover, individual training runs are estimated to generate 500 tons of CO₂ presenting environmental costs.

To read the full article, click here