How Advanced Packaging and Photonics Enable High Performance Computing

Introduction:

When examining the current landscape of AI accelerators for High-Performance Computing (HPC) or AI applications, it becomes apparent that they all share a common integration scheme.

Whether it’s GPUs, TPUs, or customized ASICs, these accelerators typically feature a configuration that involves using a chip-on-wafer substrate to bring together advanced silicon, primarily at 5-nanometer technology, and high-bandwidth memory.

Improvement On the Existing Tech Platform

However, looking towards the future, it’s evident that this platform requires significant enhancements to meet the evolving demands of high-performance computing. At the core of this need for improvement lies the requirement for higher density and lower energy compute capabilities. Achieving this necessitates the vertical stacking of multiple advanced silicon pieces to achieve the necessary computation density.

Additionally, addressing the demand for increased memory bandwidth involves incorporating more High-Performance Memory (HPM) into the package, which in turn requires further expansion of the interposer or chip-on-substrate.

Despite these advancements, challenges persist, particularly in power delivery. To overcome this challenge, integrating voltage regulators into the package becomes imperative. Moreover, issues concerning input/output (IO) and interconnect density also arise, highlighting the necessity of incorporating silicon photonics into the package.

While there are still challenges to overcome, such as bringing copackaged optics closer to the electronic die, the trajectory of technological advancement suggests that this integrated approach is the future of high-performance computing and AI accelerators.

Advanced Packaging

The rationale behind adopting stacking techniques lies in the pursuit of achieving high interconnect density or chip-to-chip communication. Illustrated by two curves, the top one represents monolithic interconnect density, while the bottom depicts conventional packaging density. The middle curve signifies the use of a chip-on-wafer (CO) solution, often referred to as a 2.5D solution.

However, transitioning to a 3D solution involves aggressively scaling the bonding pad pitch down to a few micrometer range. This approach enables the attainment of interconnect density comparable to monolithic designs, thus positioning 3D stacking as the future of semiconductor technology.

Silicon Photonics:

In discussing the future of high-performance computing, the integration of silicon photonics or co-packaged optics emerges as a significant consideration. While electrons excel at computation, photons offer superior signaling and communication capabilities. Using a 50 terabits switch as an example, currently reliant solely on electron-based systems, consumes up to 2400 Watts.

However, by implementing so-called pliable optical transceivers at the board level, power consumption can be reduced by 40%. Nonetheless, future demands necessitate even higher-speed signaling and increased bandwidth, prompting the need to convert electrons to photons earlier in the packaging process.

This transition to photon capabilities within the package, known as co-packaged optics, requires advanced stacking technologies to seamlessly integrate photonic and electronic components. By doing so, power consumption can be halved, achieving approximately five picojoules per bit. This trajectory toward co-packaged optics represents the future direction of high-performance computing.

Moreover, while current transistor technologies enable the packing of approximately 100 billion transistors on a single die, this may not suffice for future AI and machine learning applications. To address these evolving computational needs, leveraging advanced 3D packaging technologies becomes imperative to significantly increase transistor count and meet future computational demands.

Future Challenges and Solutions:

Peering into the future, Dr. Zhang outlined the challenges awaiting the semiconductor industry, particularly in the realm of high-performance computing. To meet the demands of tomorrow’s computing needs, solutions like 3D stacking and silicon photonics are imperative. These technologies promise higher interconnect density and communication capabilities, ensuring that the semiconductor industry continues to be at the forefront of technological advancement.

Conclusion:

In conclusion, Dr. Kevin Zhang’s discourse painted a compelling narrative of an industry poised for remarkable growth and innovation. The semiconductor industry, driven by advancements in technology and visionary business models, is not only meeting the needs of today but actively shaping the future. As we stand at the cusp of a technological revolution, the role of semiconductors has never been more critical, and the insights shared by Dr. Zhang serve as a guiding light for the exciting journey ahead.