The AI Boom Is Fueling a Need for Speed in Chip Networking

The new era of Silicon Valley works through networking, not the networking found on LinkedIn.

As the tech industry invests billions in AI data centers, chipmakers large and small are ramping up innovation around technology that connects chips to other chips and server racks to other server racks.

Networking technology has been around since the dawn of the computer, critically connecting mainframe computers so that they can share data. In the world of semiconductors, networking plays a role at almost every level of the stack, from the interconnection between transistors on the chip itself to the external connections made between packages or racks of chips.

Chip giants like Nvidia, Broadcom, and Marvell already have a well-established bona fide network. But in the face of the AI ​​boom, some companies are looking for new networking approaches that would help them accelerate the massive amounts of digital information flowing through data centers. This is where high-tech startups like Lightmatter, Celestial AI and PsiQuantum come in, using optical technology to accelerate high-speed computing.

Optical technology, or photonics, is having a coming-of-age moment. The technology was considered “lame, expensive and of little use” for 25 years until the AI ​​boom revived interest in it, according to PsiQuantum co-founder and chief scientific officer Pete Shadbolt. (Shadbolt appeared on a panel co-hosted by WIRED last week.)

Some venture capitalists and institutional investors, hoping to capture the next wave of chip innovation or at least find a suitable acquisition target, are investing billions in startups like these that have found new ways to accelerate data throughput. They believe that traditional interconnect technology, which relies on electrons, simply cannot meet the growing need for high-bandwidth AI workloads.

“If you look back, networking was really boring to cover because it was about changing packets of bits,” says Ben Bajarin, a longtime technology analyst and CEO of research firm Creative Strategies. “Now, with AI, it has to move pretty robust workloads, and that’s why we’re seeing innovation in speed. »

Big chip energy

Bajarin and others credit Nvidia for being prescient about the importance of networking when it made two key tech acquisitions years ago. In 2020, Nvidia spent nearly $7 billion to acquire Israeli company Mellanox Technologies, which makes high-speed networking solutions for servers and data centers. Shortly after, Nvidia purchased Cumulus Networks to power its Linux-based software system for computer networks. This was a turning point for Nvidia, which rightly bet that the GPU and its parallel computing capabilities would become much more powerful when packaged with other GPUs and installed in data centers.

While Nvidia dominates in the field of vertically integrated GPU stacks, Broadcom has become a key player in the field of custom chip accelerators and high-speed networking technology. The $1.7 trillion company works closely with Google, Meta and, more recently, OpenAI, on chips for data centers. It is also at the forefront of silicon photonics. And last month, Reuters reported that Broadcom was preparing a new networking chip called Thor Ultra, designed to provide a “critical link between an AI system and the rest of the data center.”

During its earnings call last week, semiconductor design giant ARM announced plans to acquire networking company DreamBig for $265 million. DreamBig makes AI chipsets (small modular circuits designed to be packaged into larger chip systems) in partnership with Samsung. The startup has “interesting intellectual property… which [is] “This means connecting components and sending data up and down a single chip cluster, as well as connecting chip racks to other racks.)

Light on

Nick Harris, CEO of Lightmatter, pointed out that the amount of computing power needed for AI is now doubling every three months, much faster than Moore’s Law requires. Computer chips are getting bigger and bigger. “Any time you’re at the cutting edge of the biggest chips you can build, all the subsequent performance comes from bonding the chips together,” Harris says.

His company’s approach is forward-thinking and does not rely on traditional network technology. Lightmatter builds silicon photonics that connect chips together. It claims to create the world’s fastest photonic engine for AI chips, essentially a 3D stack of silicon linked by light-based interconnect technology. The startup has raised more than $500 million over the past two years from investors like GV and T. Rowe Price. Last year, its valuation reached $4.4 billion.