Why is AI making computers and games consoles more expensive?

The AI ​​industry’s latest hot commodity is computer memory, and the sector is signing deals directly with manufacturers for billions of dollars’ worth of chips — the same chips consumers use in smartphones, laptops and gaming consoles. At best, it raises prices, and at worst, it causes shortages that limit production.

Why does AI need so much memory?

AI models are very, very big. You can think of them as grids of billions, or even trillions, of parameters – numbers stored in memory – on which extremely repetitive but, taken as a whole, demanding calculations are performed. This is how a large language model takes input and generates output.

Transferring this amount of data to processors from cheap but slow hard drives – what we commonly call computer storage – creates absurd bottlenecks. To avoid this, huge amounts of much faster RAM – what we normally call computer memory – are used instead.

And there’s another factor: the models created by AI companies operate on an enormous scale. This means that they require computers capable of running hundreds, thousands, or millions of copies of these models, so that a large number of clients can use them at the same time.

Take an extremely computationally intensive task, scale it to a large number of users, remove the limits on expansion by adding virtually infinite investments into the mix, and you get an insatiable demand for hardware. A company that makes a few million laptops a year just isn’t up to the task.

Why can’t chipmakers just make more chips?

It’s easier said than done. Semiconductor factories have limited capacity and building a new one involves massive investment and often takes several years.

There are also signs that manufacturers are unwilling to end the drought. Korean media reports that Samsung Electronics and SK Hynix, which together make about 70% of these chips, are reluctant to increase supply too much in the event of a downturn in the AI ​​industry and are left with unused, expensive new chip factories and a shortfall in orders.

And with current demand booming and Samsung in the comfortable position of being able to raise prices by up to 60%, why would the company rock this boat? The figures show that a 32GB chip that Samsung was selling for $149 in September was on sale for $239 in November.

Have we seen shortages like this before?

Again and again. For years, the AI ​​boom has seen companies suck up every possible graphics processing unit (GPU) computer chip to build vast data centers capable of training and running ever-larger models. This relentless demand is why chipmaker Nvidia’s stock price has risen from $13 in early 2021 to a high of more than $200 in recent months.

In 2021, we experienced shortages of all kinds of computer chips due to a perfect combination of factors, including the global pandemic, a trade war, wildfires, drought, and snowstorms. This has affected the manufacturing of everything from pickup trucks to microwaves.

We even saw a shortage of hard drives that same year when a new cryptocurrency called Chia, which ran on storage space rather than computer power, gained popularity.

In short, technology evolves quickly. Sometimes much faster than global supply chains.

When will the shortage end?

Not soon. OpenAI has signed agreements with Samsung and SK Hynix that will allow it to take delivery of approximately 40% of the world’s memory supply. And it’s just one AI company, even if it’s one of the giants. Microsoft, Google and ByteDance, among others, are also buying every chip possible.

One way to end the shortage – and perhaps quickly create a glut – would be for the AI ​​collapse that economists, bankers and even the head of OpenAI are warning to actually happen. But this would likely have devastating economic consequences, so it may not be a panacea.

If this collapse doesn’t happen, estimates suggest it may be until 2028 before things calm down and demand and supply return to balance, with some smaller companies bringing new factories online.

Some suggest that this expectation could be a problematic leak for the entire manufacturing industry. Sanchit Vir Gogia, an industry analyst at Greyhound Research, told Reuters that “memory shortage has now moved from a component-level concern to a macroeconomic risk.”