Connect with us

Tech

AI and the High Bandwidth Memory Shortage

Published

on

While browsing our website a few weeks ago, I stumbled upon “How and When the Memory Chip Shortage Will End” by Senior Editor Samuel K. Moore. His analysis focuses on the current DRAM shortage caused by AI hyperscalers’ ravenous appetite for memory, a major constraint on the speed at which large language models run. Moore provides a clear explanation of the shortage, particularly for high bandwidth memory (HBM).

As we and the rest of the tech media have documented, AI is a resource hog. AI electricity consumption could account for up to 12 percent of all U.S. power by 2028. Generative AI queries consumed 15 terawatt-hours in 2025 and are projected to consume 347 TWh by 2030. Water consumption for cooling AI data centers is predicted to double or even quadruple by 2028 compared to 2023.

But Moore’s reporting shines a light on an obscure corner of the AI boom. HBM is a particular type of memory product tailor-made to serve AI processors. Makers of those processors, notably Nvidia and AMD, are demanding more and more memory for each of their chips, driven by the needs and wants of firms like Google, Microsoft, OpenAI, and Anthropic, which are underwriting an unprecedented buildout of data centers. And some of these facilities are colossal: You can read about the engineering challenges of building Meta’s mind-boggling 5-gigawatt Hyperion site in Louisiana, in “What Will It Take to Build the World’s Largest Data Center?

We realized that Moore’s HBM story was both important and unique, and so we decided to include it in this issue, with some updates since the original published on 10 February. We paired it with a recent story by Contributing Editor Matthew S. Smith exploring how the memory-chip shortage is driving up the price of low-cost computers like the Raspberry Pi. The result is “AI Is a Memory Hog.”

The big question now is, When will the shortage end? Price pressure caused by AI hyperscaler demand on all kinds of consumer electronics is being masked by stubborn inflation combined with a perpetually shifting tariff regime, at least here in the United States. So I asked Moore what indicators he’s looking for that would signal an easing of the memory shortage.

“On the supply side, I’d say that if any of the big three HBM companies—Micron, Samsung, and SK Hynix—say that they are adjusting the schedule of the arrival of new production, that’d be an important signal,” Moore told me. “On the demand side, it will be interesting to see how tech companies adapt up and down the supply chain. Data centers might steer toward hardware that sacrifices some performance for less memory. Startups developing all sorts of products might pivot toward creative redesigns that use less memory. Constraints like shortages can lead to interesting technology solutions, so I’m looking forward to covering those.”

To be sure you don’t miss any of Moore’s analysis of this topic and to stay current on the entire spectrum of technology development, sign up for our weekly newsletter, Tech Alert.

From Your Site Articles

Related Articles Around the Web

>

Continue Reading

Tech

Planned Amazon data center could become the biggest climate polluter in the U.S.

Published

on

As part of a planned data center in Pecos County, Texas, Amazon is investing in an on-site power plant that could become the largest source of climate pollution in the United States, according to The New York Times.

The NYT says the plant would burn natural gas and is permitted to release 33 million tons of carbon dioxide per year — more than any other power plant in the U.S.

In a statement, an Amazon spokesperson confirmed that the data center will “be powered by new on-site generation that won’t raise electricity costs for Texas families.” (Data centers face growing political opposition for a number of reasons, including their effect on electricity costs.)

AI has already had a significant impact on Amazon’s carbon emissions, which it reported were up 16% last year — the wrong direction for a company that pledged to eliminate its carbon emissions by 2040. And that could get worse as Amazon and tech companies back the development of huge natural gas plants to support their power-hungry data centers.

The Amazon spokesperson said, “The world looks different now than when we co-founded the climate pledge,” while also claiming, “Our commitment hasn’t changed.”

>

Continue Reading

Tech

OpenAI acquires presentation startup NextSlide

Published

on

NextSlide recently announced that it’s joining OpenAI, with the presentation startup’s team members now working on ChatGPT.

The NextSlide website currently displays a note from founder Ahmed Beshry describing the startup’s product as one “that could turn prompts, notes, documents, or research into a polished, editable presentation.”

The ultimate goal, Beshry said, was “to make visual communication more accessible and help more people express their ideas clearly.” So by joining OpenAI, the team will “continue pursuing that same mission: building AI products that help people create, communicate, and turn their ideas into meaningful work.”

The financial terms of the deal were not disclosed. In a note on LinkedIn, Beshry said the announcement is coming “a few months late,” as the acquisition took place “earlier this year.”

Beshry was previously a co-founder at Caper AI, a smart cart/cashier-less checkout startup acquired by Instacart in 2021.

>

Continue Reading

Tech

X replaces ‘misaligned’ revenue sharing program with Original Content Rewards

Published

on

X, the social media platform now owned by Elon Musk’s SpaceX, is shaking up how it pays influencers and creators.

In announcing the change, the company said it will be winding down its existing Revenue Sharing program and replacing it with something called Original Content Rewards. X will stop accepting new Revenue Sharing participants, while existing participants will continue earning money through September 7.

Then, starting on September 8, they’ll be able to apply for the new program. Participants will still need to subscribe to one of X’s Premium tiers, and there will be qualifying thresholds for follower count (500 verified followers) and impressions (500,000 Home Timeline impressions from verified users in 90 days), but it sounds like the big change is the emphasis on originality. 

What counts as original content? X said it can include original reporting and analysis, photos and videos created by the poster, or memes and graphics they’ve designed themselves. Commentary also counts, but “if your content regularly incorporates material created by others, you’ll need to contribute meaningful original value for it to qualify under our original content guidelines.”

The company also included examples of posts that won’t count as original, such as those just copied over from another account, downloaded from one account and re-uploaded to your own, or reposting content “without meaningful transformation.”

This announcement follows repeated attempts by X to reform the Revenue Sharing program, for example reducing payments to aggregators and “clickbait” accounts in April. But these efforts have also prompted complaints from popular accounts profiting from the current system; Musk even reversed some of those changes (giving a creator’s local audience more weight when calculating payouts) after a backlash.

In a post about the new changes, X’s Allegra Jacchia wrote that the existing program “had reached a point where its incentives were misaligned.”

“Creators should be focused on bringing net new content to the platform instead of maximizing payouts,” she said. “We could have kept adding more rules and exceptions, but ultimately the better decision was to start fresh and build a program designed from day one to reward originality.”

Jacchia added that X be “continue refining the program, improving our models, and raising the bar over time.”

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

>

Continue Reading

Trending

Copyright © 2017 Zox News Theme. Theme by MVP Themes, powered by WordPress.