Arm Total Design Ignites Growing Ecosystem of Arm-based Silicon for a Sustainable AI Datacenter

By Eddie Ramirez, VP, Infrastructure Business, Arm

October 16, 2024 -- Datacenters are constantly challenged to balance power demands with the growth of AI workloads, the increasing cost and complexity of developing chips, and the need for sustainability.

We introduced Arm Total Design a year ago to address these challenges by creating an ecosystem of partners to accelerate the development of custom silicon, bringing together key industry players to build solutions for the datacenters of the future with Arm Compute Subsystems (CSS). It has quickly grown into a multivendor, Arm-based chiplet and SoC ecosystem, bringing together capabilities from design through to foundry manufacturing and doubling to nearly 30 participating companies, with Alcor Micro, Egis, PUF Security and SEMIFIVE as the latest companies to join the ecosystem.

New Arm-based solutions to power the AI datacenter sustainably

Arm Total Design has sparked global collaboration, leading to real-world CSS-powered solutions for GenAI computing. A fantastic example is the news today that Arm, Samsung Foundry, ADTechnology and Rebellions are partnering to bring to market an AI CPU chiplet platform. This platform, targeting cloud, HPC, and AI/ML training and inference workloads combines Rebellions’ REBEL AI accelerator built with a Neoverse CSS V3-powered compute chiplet from ADTechnology to be implemented with Samsung Foundry 2nm Gate-All-Around (GAA) advanced process technology. It promises to deliver unparalleled performance and optimal power efficiency, with an estimated 2-3x efficiency advantage for GenAI workloads (Llama3.1 405B parameter LLMs).

“AI and HPC designs require technology solutions that deliver maximum performance, high transistor density, and energy efficiency,” said Taejoong Song, Vice President and Head of Foundry Business Development at Samsung Electronics. “Samsung Foundry’s 2nm GAA process is designed precisely to satisfy the most stringent HPC and AI design requirements, and we’re excited to leverage the flexibility of Arm CSS and the power of the Arm Total Design ecosystem to deliver an AI CPU chiplet platform, which will further accelerate adoption of our leading-edge technology and design solutions for hyperscalers and cloud service providers.”

This exemplifies the unique value of Arm Total Design and standards-based compute subsystems in accelerating AI silicon development by integrating Arm-optimized EDA tools, global design expertise and foundry partnerships to facilitate easy integration by AI accelerator designers. With the rapid evolution of AI workloads, tightly coupled CPU compute is essential for supporting the complete AI stack. Data pre-processing, orchestration, database augmentation techniques, such as Retrieval-augmented Generation (RAG), and more all benefit from performance-efficiency of Arm Neoverse CPUs. We’ve baked support for these requirements into our CSS and through Arm Total Design, the ecosystem is already benefitting from these innovations.

A new standard for purpose-built AI infrastructure

CSS and Arm Total Design are helping to create the hardware foundation for a sustainable AI datacenter. Arm Total Design is already accelerating the development of Arm-based test chips and chiplet products powered by Neoverse N-series or V-series CSS. The diversity of chiplet solutions spanning cloud to edge and the pace at which they are being developed is a direct result of reducing barriers to entry by enabling broad, preferential access to the latest CSS.

Just today Alcor Micro announced they are building a chiplet powered by CSS and targeting AI/ML training and inference use-cases. And recently Alphawave announced its own advanced compute chiplet built on CSS for AI/ML, HPC, datacenter and 5G/6G applications. These Arm-based chiplets exemplify the diversity, flexibility and global supply chain that only the Arm partnership can deliver.

Additionally, Arm Total Design partners including Alphawave, Cadence, and proteanTecs, are validating their third-party IP products with CSS on advanced nodes to ensure compliance with Arm specifications and standards. This means partners can build CSS-based custom silicon on leading edge nodes and have a seamless out-of-the-box software experience.

This type of software readiness remains a critical gateway to the potential of AI and for more than 30 years we have invested in ensuring software on Arm “just works”. Today all major frameworks and OSes run on Arm. For the Arm Total Design ecosystem, this means the incredibly diverse set of silicon solutions our partners are bringing to market can leverage an equally vibrant and cohesive software ecosystem. One of the latest examples of this ongoing investment is the introduction of Arm Kleidi technology which optimizes CPU-based inference on Arm to open source projects like PyTorch and Llama.cpp. This is especially important for our Arm Total Design partners who are building CSS-based chiplets for edge AI computing without the need for an accelerator.

Driving the next chapter of silicon innovation

As AI compute demands increase, it’s vital for developers to easily run their innovations on the most ubiquitous compute platform on the planet in a performant, power-optimized and accessible way. CSS and Arm Total Design accelerate ecosystem AI development by enabling faster access to hardware and software advancements. We are in a new chapter of engineering creativity and development, and Arm is delivering the tools and technologies to drive AI and silicon innovation where they need to go.

Discover more about Arm Total Design and its role in the future of AI-powered computing at OCP where you can join us for talks, demos, and partner showcases to learn about the next era of built-for-AI silicon – on Arm.