How the Rise of Data Center Disaggregation Creates Demand for Co-Packaged Optics

May 24, 2022

Blog

From online transactions to streaming videos and big data analytics, data centers are proving to be the workhorses of our smart, connected world. Higher volumes of data coupled with increasingly complex data are causing a shift in data center architectures. A new trend has emerged in data center architectures to address these two underlying forces: data center disaggregation.

To support more efficient processing of massive data workloads, disaggregated data centers are marked by compute, networking, storage, and optics resources that are separated in different boxes and connected optically.

Let’s examine the changes in data center architectures and how optical technologies can facilitate these changes.  

Unstoppable Demand for High Bandwidth and Low Latency

Cloud computing is extending its reach across a variety of industries, including chip design. Meanwhile, companies in data-intensive areas like software platforms, e-commerce, and social media are building their own hyperscale data centers. Inside these centers are several thousand to tens of thousands of servers, working diligently to support the functions and transactions that allow us to watch movies, buy groceries, and work from our mobile devices. To provide an idea of where data demands are headed, have a look at these assessments from the IEEE 802.3 Ethernet Bandwidth Assessment Report:

• By 2025, an anticipated 38 billion devices will be connected to the internet, up from about 29 billion this year
• Between 2017 and this year, there will be a projected 200% increase in average traffic per user and per household
• Video-based data is expected to grow from 75% of data (roughly 90 exabytes per month) in 2017 to 82% of data (roughly 325 exabytes per month) this year

Optical Interconnects for Faster Data Transmission  

Disaggregated data center architectures are ideal for meeting endless demands for high bandwidth and low latency. In this approach, optical interconnects connect homogeneous resources, providing better flexibility, higher density, and better utilization. When a workload comes in, a central intelligence unit figures out and takes only what is needed from the compute, networking, and storage resources, thus eliminating any waste. Remaining resources can then be directed to other jobs.

Optical interconnects transmit signals via light. Compared to their copper counterparts, optical interconnects support higher bandwidth and speed, lower latency, and lower power. They’ve been proving their worth for rack-to-rack, room-to-room, and building-to-building configurations. With their pluggable modules, using optical interconnects also makes it easier to upgrade network infrastructures to support 400G, 800G, and 1.6T Ethernet.

With data network speeds surpassing 400 Gbps, many engineers are worried about how much power is needed to drive the electrical signals to the optical modules. Co-packaged optics, composed of a single package integrating electrical and photonic dies, can help. The electrical link between the host SoC and the optical interface is connected to co-packaged optics in the package rather than to the pluggable module in the faceplate of a server rack, making the link shorter and, therefore, more power efficient.

The Role of Die-to-Die Interface IP

When a system features co-packaged optics, the optical interconnects must support multi-chip modules (MCMs). MCMs rely on die-to-die controllers and PHY for connectivity. These controllers—which need to deliver efficient inter-die connectivity in high-performance computing, server, and networking SoCs—should ideally be optimized for latency, bandwidth, power, and area. The PHY, meanwhile, comes in different formats depending on the needs.

Many designers use long-reach PHY on copper interconnects, but these are starting to reach their limits, especially for large SoCs that have hundreds of PHY lanes. This is leading some engineers to go with very short reach (VSR) PHY for pluggable optical modules.

With the growing popularity of co-packaged optical modules, extra-short-reach (XSR) PHY and, looking ahead, Universal Chiplet Interconnect Express (UCIe) PHY will no doubt play important roles, too. Both of these formats allow placement of the optics chip very close to the host chip or on the same package substrate.

Data makes our digital world go ‘round and, in answer to our insatiable appetite for data, disaggregated data centers are emerging as a popular architecture. Optical interconnects are their highways, helping to ensure that we can uncover useful insights through complex modeling, stream high-definition programs from our smartphones, and engage in a variety of other online activities smoothly and swiftly.