Everything in their statement can be true and it be a bad thing for consumers of non-HBM RAM.
"HBM capacity to expand to 250,000 wafers a month"
So let's say their current HBM capacity is 100k wafers/month (pure speculation/random number for illustration), and their total RAM capacity (including HBM( is 300k wafers/month, then non-HBM capacity reduces from 200k to 50k.
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
I agree that datacenters are not going to go away, but I have doubts that the buildup that has happened is really going to pay off for most operators.
But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”
Part of the reason harnesses work well is you can run a lot of agents in parallel. That doesn't slow down demand.
LLMs are amazing tech, but they're terrible without oversight. More agents faster just makes reality collapse on them quicker.
But yeah, you're right, temporarily, this will still push demand. But the topic was about "diminishing returns" as in "tech getting better". Not as in "customer spending".
Before that, we had 0. After that, we had more than 1.
A leap as far as that is hard to recreate.
But that wasn't my point. That's just trolling.
The actual point is that LLMs aren't gaining new capabilities anymore. They just get more reliable at the ones they already have; turning what was a coin flip to some higher probability.
That's (intuitively speaking, not strictly mathematically speaking) kinda the mathematical definition of diminishing returns.
It may turn out that demand for SOTA frontier models isn’t so limitless after all.
Their choice, their consequences.
I expect something similar for AI. It's useful tech... just not very profitable tech, and certainly not to the tune of trillions of dollars worth of public demand. The promises of superintelligence, replacing everyone, and the rest of the hype will die with the companies who made the promises, but the tech will survive and thrive.
Which governments, and what exactly would you want these governments to do? The demand is global. There are no levers a single government can pull to meaningfully influence the global demand without fully committing to an protectionist economic policy, in which case the U.S. doesn't have the facilities to magically pop up world class fabs overnight, and South Korea and Taiwan don't have the market demand that the U.S. generates to justify their investments in making these chips and China lacks the IP to be able to build anything comparable to Nvidia's silicon at the moment.
No one has all the cards and no one controls all the levers.
Especially not for consumer goods.
Can that share of production be allocated to consumers, and the AI fights over the rest?
I'm as fanatical a free-market fundamentalist as you'll ever meet, but if someone were to argue that government has a role in preventing bullshit like OpenAI's unilateral 40% attack on the entire DRAM market, backed by nothing but funny money, I would have a hard time coming up with defensible counterarguments.
Alternatively, the money invested can be rational because it prices out competitors and establishes a monopoly, after which point the monopolist earns their absurd investment back with complete control of the market. This is also bad.
The problem you have here is hundreds of different companies are doing the "gambling" in a non collusionary manner so it's going to take decades in court to prove it. Are you saying governments should do an authoritarian take over of RAM allotment?
https://news.skhynix.com/en/fab-facility-investment-2026/
It's not an overnight thing obviously.
HBM is meant to be integrated into the same package as the CPU, so no more DIMM sockets. It also has higher latency apparently.
People will have to get used to buying a fixed amount of RAM with their CPU but thats unlikely to be a problem.
They have managed to pull this sort of thing off many many times. https://en.wikipedia.org/wiki/Reality_distortion_field
If I get 10% more performance for 50% more cost it really depends on one's needs, for example.
This is really not a limit because of unified memory -- in principle, PCIe GPUs could read/write main memory without the CPU. But it's a limit for /fast/ unified memory, because fast means close.
So unified memory is great as long as the integrated GPU is strong enough. Then it has two advantages: a) probably faster transfer CPU<->GPU (but that's an implementation choice for the non-unified case b) If you either need a lot of memory for the CPU or the GPU, but not for both at the same time, you pay for memory only once.
No, Mac laptops use LPDDR, currently LPDDR5X.