What is ARM Neoverse: the IP behind current HPC and datacenter workloads

Introduction

ARM’s IPs have emerged as crucial foundation over which modern day processors are built. ARM’s Neoverse platform is an example of one such IP. Neoverse is specially focused on HPC, data center and Edge AI workloads. As the foundation for a trillion intelligent devices, spanning smartphones, IoT, servers, and beyond, Neoverse provides the essential building blocks for efficient, secure, and scalable infrastructure solutions. In this article, we’ll delve into what ARM Neoverse is, explore its features, and highlight a real-world example of a big tech giant leveraging Neoverse for their custom processors.

How ‘Chips’ Become ‘ARM-Based’

When we talk about “ARM-based” chips, we’re referring to processors that utilize ARM’s architecture. Here’s how it works:

ARM Architecture: ARM provides a set of standardized instructions and design principles for building processors. These instructions define how the CPU executes tasks, manages memory, and communicates with other components.
IP Licensing: Companies license ARM’s IP to create their custom processors. They can choose from various ARM cores, interconnects, and other components to build a chip tailored to their specific needs.
Customization: Chip designers add their own logic, accelerators, and optimizations alongside ARM’s IP. This customization allows them to create specialized processors optimized for specific workloads.
Fabrication: Semiconductor manufacturers fabricate the chip once they complete the design. These ARM-based chips are utilized in servers, networking equipment, and other infrastructure devices.

ARM Neoverse Features

ARM Neoverse N3

The extension of the N-series portfolio focuses on power efficiency, making it ideal for applications requiring high efficiency such as telecommunications, networking, and data processing units.

The Neoverse N3 CPU offers impressive per-die specifications, including up to 32 cores of ARM Neoverse N3, up to 4 channels (40b) of DDR5/LPDDR5 memory, up to 32 lanes of PCIe Gen5 or CXL I/O, and high-speed die-to-die links supporting UCIe 1.1 or Custom PHYs. This architecture is designed to cater to the demands of power-sensitive workloads with efficient performance.

ARM Neoverse V1

Renowned for its high single-threaded performance, the ARM Neoverse V1 specifically targets cloud and HPC workloads. Equipped with features like Scalable Vector Extensions, the Neoverse V1 optimizes for processing intensive tasks. The V1 architecture supports top-down level 1 metrics, providing a comprehensive set of events and metrics for performance analysis and optimization, making it a valuable asset for a range of processing-intensive applications.

ARM Neoverse S3

The Neoverse S3 represents the third generation of infrastructure-specific system IP, serving as an ideal foundation for next-generation infrastructure System-on-Chips (SoCs). These SoCs find applications across a wide spectrum, ranging from high-performance computing (HPC) and machine learning down to edge devices and display processing units.

Neoverse S3 supports upgraded IO technologies, including PCIe Gen6, CXL 3.1, DDR5, and HBM3
Neoverse S3 uses Arm’s AMBA CHI C2C over UCIe (Unified Chiplet Interface), for seamless communication between different chiplets within an SoC.
Neoverse S3 introduces Arm Realm Management Extension (RME), enabling confidential computing. RME ensures that data remains encrypted even while “in use” in memory

ARM Neoverse Compute Subsystems

Arm Neoverse Compute Subsystems (CSS) integrate and verify platforms that bring together all the key components necessary for the heart of a system-on-chip (SoC). These subsystems accelerate the development of custom silicon for infrastructure workloads, including cloud computing, high-performance computing (HPC), and artificial intelligence (AI) acceleration.

By providing a pre-configured, validated, and optimized starting point, Neoverse CSS enables Arm partners to develop custom chips more efficiently.

Components of Neoverse CSS:
- Neoverse Cores: These are the fundamental processing units within the subsystem. Depending on the specific CSS version, they can be Neoverse V3, N3, or N2 cores.
- CMN Mesh (Coherent Mesh Network): The CMN interconnect ensures cache coherence and efficient communication between cores, memory, and I/O components.
- System IP: Neoverse CSS includes essential system-level components such as memory controllers, I/O interfaces, and system management features.
- Software and Development Tools: Arm provides software support, firmware, and development tools to help partners optimize their designs.

Real-World Example: Big Tech Giant Leveraging Neoverse

AWS Graviton

Amazon Web Services custom-designs Graviton processors using Arm architecture and utilizes Arm Neoverse cores. These processors are optimized to deliver high performance and energy efficiency in cloud environments.

AWS Graviton3, the latest iteration, provides up to 25% better compute performance compared to its predecessor. AWS Graviton3 is built using the Arm Neoverse V1 core. This core design focuses on performance, scalability, and efficiency. It enables AWS to offer instances with improved performance while maintaining energy efficiency.

Google’s Axion CPU

Google Axion Processors represent Google’s custom Arm-based CPUs designed specifically for data centers. These processors combine Google’s silicon expertise with Arm’s highest-performing CPU cores. Axion processors are built using the Arm Neoverse V2 CPU.

The Neoverse V2 core delivers giant leaps in performance for general-purpose workloads, including web and app servers, data analytics engines, and CPU-based AI training and inferencing.

Axion processors outperform the fastest general-purpose Arm-based instances available in the cloud today.

They offer up to 30% better performance, up to 50% better performance, and up to 60% better energy efficiency compared to comparable current-generation x86-based instances.

Conclusion

ARM Neoverse powers trillion devices, spanning smartphones, IoT, servers, and beyond. Its flexible architecture and advanced features are now driving innovation in HPC and data centers. Real-world examples like AWS Graviton and Google’s Axion CPU showcase its impact, shaping the future of computing by marrying performance with energy efficiency across diverse applications.