Spiking Transformer Hardware Accelerators in 3D Integration
By Boxun Xu1, Junyoung Hwang2, Pruek Vanna-iampikul2, 3, Sung Kyu Lim2, Peng Li1
1 Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA, USA
2 Department of Electrical and Computer Engineering, Georgia Institute of Technology, GA, USA
3 Department of Electrical Engineering, Burapha University, Chonburi, Thailand
Spiking neural networks (SNNs) are powerful models of spatiotemporal computation and are well suited for deployment on resource-constrained edge devices and neuromorphic hardware due to their low power consumption. Leveraging attention mechanisms similar to those found in their artificial neural network counterparts, recently emerged spiking transformers have showcased promising performance and efficiency by capitalizing on the binary nature of spiking operations. Recognizing the current lack of dedicated hardware support for spiking transformers, this paper presents the first work on 3D spiking transformer hardware architecture and design methodology. We present an architecture and physical design co-optimization approach tailored specifically for spiking transformers. Through memory-on-logic and logic-on-logic stacking enabled by 3D integration, we demonstrate significant energy and delay improvements compared to conventional 2D CMOS integration.
To read the full article, click here
Related Chiplet
- 12nm EURYTION RFK1 - UCIe SP based Ka-Ku Band Chiplet Transceiver
- Interconnect Chiplet
- Bridglets
- Automotive AI Accelerator
- Direct Chiplet Interface
Related Technical Papers
- 2D materials-based 3D integration for neuromorphic hardware
- Co-Optimization of Power Delivery Network Design for 3-D Heterogeneous Integration of RRAM-Based Compute In-Memory Accelerators
- Advancing Trustworthiness in System-in-Package: A Novel Root-of-Trust Hardware Security Module for Heterogeneous Integration
- Material-Mechanistic Interplay in SiCN Wafer Bonding for 3D Integration
Latest Technical Papers
- AuxiliarySRAM: Exploring Elastic On-Chip Memory in 2.5D Chiplet Systems Design
- System-Level Validation Across Multiple Platforms to build a Robust 2.5D Multi Foundry Chiplet Solution
- Material-Mechanistic Interplay in SiCN Wafer Bonding for 3D Integration
- Fault Modeling, Testing, and Repair for Chiplet Interconnects
- Low-Loss Integration of High-Density Polymer Waveguides with Silicon Photonics for Co-Packaged Optics