Whilst current models can't 'intuit' and come up with conjectures, they can certainly disprove some of them very quickly through the kind of grind that humans can't do. I suppose there really are some mathematicians out there today, whose last few years of study, have just been up-ended by this.
--
"Yes we are," insisted Majikthise. "We are quite definitely here as representatives of the Amalgamated Union of Philosophers, Sages, Luminaries and Other Thinking Persons, and we want this machine off, and we want it off now!"
"What's the problem?" said Lunkwill.
"I'll tell you what the problem is mate," said Majikthise, "demarcation, that's the problem!"
"We demand," yelled Vroomfondel, "that demarcation may or may not be the problem!"
"You just let the machines get on with the adding up," warned Majikthise, "and we'll take care of the eternal verities thank you very much. You want to check your legal position you do mate. Under law the Quest for Ultimate Truth is quite clearly the inalienable prerogative of your working thinkers. Any bloody machine goes and actually finds it and we're straight out of a job aren't we? I mean what's the use of our sitting up half the night arguing that there may or may not be a God if this machine only goes and gives us his bleeding phone number the next morning?"
>I don’t understand it yet. Maybe it’ll take me an afternoon to check all the calculations, but what would still be missing is why this was an approach that would’ve made sense in the first place. Is there some broader context or theory within which this would’ve been the obvious thing to do? What other results can be proven using these techniques? What is it telling us about quantum information or operator theory? I have no idea. I spent about an hour this morning asking ChatGPT these questions, but it’s somewhat frustrating because it speaks with a mishmash of physicist, operator algebraist, quantum information theorist-lingo, plus the usual LLM breezy lilt that annoys everybody.
They certainly seem to have "intuited", in a way that is not immediately obvious to experts in the field, the way to solve at least some of these problems. This was not just simply grinding away at a method that humans already knew would work and just hadn't gotten to yet.
[0] https://nitter.poast.org/henryquantum/status/208362369543662...
The waveform collapse is natively "very high dimensional", not sure why the "but" part of the sentence belongs here.
Quantum waveform collapse has been proposed as explanation for consciousness, allowing to explain how an entity can have free will and yet obey rigid physical laws: https://en.wikipedia.org/wiki/Consciousness_causes_collapse
I think OP is suggesting a similar thing happened in the LLM, implying it gained consciousness despite following a well-defined compute process.
https://github.com/mxgmn/WaveFunctionCollapse
It's one of the coolest things I've ever seen.
I guess I've been pronouncing it wrong though. It's wave function collapse, not waveform.
I haven't studied it properly yet but it kind of looks like how sudokus work. You have a bunch of plausible options, and the various possible worlds overlap with each other but not completely and then you eliminate the things that are not possible until the actual possibility remains.
I think thinking works similarly, and I wouldn't be surprised if artificial thinking also worked similarly.
That's how they are finding these solutions though, unless we are just going to label intuition as something only humans can do. Like a submarine being unable to swim or whatever that example is.
The two places were seeing lots of movement are:
* Updates to lower/upper bounds. In many cases, these kinds of problems are the deep-math equivalent of calculating more digits of pi. Yes, if you throw time at it you'll break the record, but it may not be terribly worthwhile.
* Finding counter examples which disprove conjectures. This is really useful, and helps offset some positivity bias on the human side, often bringing together known tools from distant silos.
If you read the list of ten results, almost all fall into one of these buckets.
As someone with a PhD in combinatorics, I believe that I'm qualified to say that, yes, there are problems as useless as calculating more digits of pi.
There's nowhere else to move the goalposts. You've already stashed them in the far corner of the parking garage down the street from the stadium. If you go any farther you'll leave the school grounds entirely.
Matrices are an implementation detail in reconstructing the surface of human knowledge. It's a complex surface, but it's a regurgitation.
I disagree. I routinely let LLMs speculate or generate hypotheses along the way of helping with technical research. Sometimes they can prove the correctness of a concrete math idea but other times even an unproven conjecture helps with the numerical algorithm implementation and the result is then simply supported by additional data. I guess that any autoresearch-adjacent application has LLMs intuiting and coming up with hypotheses/conjectures—as do the steps/lemmas along a complex proof. In my opinion the modern LLMs are powerful intuitive thinkers that generate lots of conjectures of varying quality or importance.
Of course computers can grind in a way that humans can't. But now we have systems that convert the human-comprehensible ideas into a computer's plan of attack, in a way that greatly expands the frontier of ideas thus treatable.
"Excuse me, We demand rigidly defined areas of doubt and uncertainty!"
DT: Might I make an observation at this point?
MT: You keep out of this metal nose.
VF: We demand that that machine not be allowed to think about this problem!
DT: If I might make an observation…
MT: We’ll go on strike!
VF: That’s right. You’ll have a national philosopher’s strike on your hands.
DT: Who will that inconvenience?
MT: Never you mind who it’ll inconvenience you box of black legging binary bits! It’ll hurt, buster! It’ll hurt!
DT: [Booming] If I might make an observation …
“All I wanted to say,” bellowed the computer, “is that my circuits are now irrevocably committed to calculating the answer to the Ultimate Question of Life, the Universe, and Everything.” He paused and satisfied himself that he now had everyone’s attention, before continuing more quietly. “But the program will take me a little while to run.”
Fook glanced impatiently at his watch.
“How long?” he said.
“Seven and a half million years,” said Deep Thought.
Lunkwill and Fook blinked at each other.
“Seven and a half million years!” they cried in chorus.
“Yes,” declaimed Deep Thought, “I said I’d have to think about it, didn’t I? And it occurs to me that running a program like this is bound to create an enormous amount of popular publicity for the whole are of philosophy in general. Everyone’s going to have their own theories about what answer I’m eventually going to come up with, and who better, to capitalize on that media market than you yourselves? So long as you can keep disagreeing with each other violently enough and maligning each other in the popular press, and so long as you have clever agents, you can keep yourselves on the gravy train for life. How does that sound?”
The two philosophers gaped at him.
“Bloody hell,” said Majikthise, “now that is what I call thinking. Here, Vroomfondel, why do we never think of things like that?”
“Dunno,” said Vroomfondel in an awed whisper; “think our brains must be too highly trained, Majikthise.”
So saying, they turned on their heels and walked out of the door and into a life-style beyond their wildest dreams.”
People keep saying this. Why?
Surely the AI can complete the prompt “Generate new research questions based on these observations”?
When I read the reasoning traces of coding models they are constantly asking themselves questions and attempting to answer them.
They can't exit the hull until the "intuition" starts spawning points outside the convex hull.
It's a convex hull of information that is reflective and spans outside of itself and combines in a new way, when you shine two known rays of light together from the inside.
Now it gets better. AI can be orders of magnitude more creative than any human could ever hope for, because his convex hull of information is orders of magnitude larger, and the possibilities for new combinations are equally larger.
Not completely. Novelty used to be a major thing, when Humans did it. Another important criteria used to be if that new thing makes sense at all. Here the language model has a problem, as it doesn't have the means to evaluate this criteria.
The extent to which they are able to do this is the more interesting question!
By the way, convex hull permits extrapolating past the training data. LLM won't invent a new word that could not be defined by a sequence of known words. Just if it's meaningless and fully random/hallucinated, the new knowledge won't work with other known information blocks (breaks convexity).
Humans can “intuit” based on a much larger, if not unlimited, context. Also I just want to say that human cognition is something so insanely complex and deep that we will not understand it at all in my lifetime. To attribute all, or really any, aspects of human cognition to a machine at this point is silly to me.
Most humans are dumber than a box of rocks. Here in Seattle we had one of many light rail-related fuckups where they had to replace part of the line with buses. People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.
Chatgpt was smarter than the average person a while ago
This does not demonstrate a lack of intelligence. It demonstrates laziness and a lack of interest in spreading apart. Or just lack of consideration (or even malice) on the part of those at the back of the wad.
> Chatgpt was smarter than the average person a while ago
This is an absurd claim that fundamentally misunderstands what it means to be "smart". Reasoning that would get you to this conclusion would equally well apply to Google's search engine over a decade ago.
There are processes at work there that we don’t even have the language to describe.
(86 billion is the number ChatGPT, ironically enough, has given me a couple of times. I remember hearing for a long time that it was estimated to be somewhere in the ballpark of 100 billion. This is not my field of study.)
You can't find things on a map that aren't there, but maybe you can draw a route nobody used before.
For the same reason that you can't draw a 15 of Diamonds from a regular card deck.
LLMs have made math proofs more computable, in the sense that a computer can both generate potential solutions and check the validity of its solutions on its own, with a reasonable chance of converging on something correct. I assume this was already doable to some extent, but it seems like it’s now exponentially easier. That still doesn’t mean that all math is automatically solved.
This is somewhat similar to things like molecular dynamics or protein folding or finite element simulations, etc. Some problems that were previously intractable via computation became tractable. Others - the vast majority of other problems - remain unsolvable by these computational techniques, because the scale of compute required is beyond imagination. These are simple things like simulating the dynamics of a cubic millimeter of water molecules for 1 second. Unfathomably beyond current capabilities (and LLMs aren’t going to change that).
I think LLMs are great, I use them every day and I think they have a ton of value. But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.
So there are limits - but we’re about to learn a lot about the new normal of what constitutes a layup math proof vs the truly difficult.
I think the question, that we keep stumbling over, is what problems are computable.
> But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.
Let the goalpost shifting continue. It'll buy us another half year or so.
On the other hand even if the compute allocated by openai is esquivalent to day 10 human mathematicians, the machines can work 24h per day, that is already a lot more productive.
I’m not sure if people just aren’t as aware, but the Jacobian conjecture practically was on par with those other great problems.
By definition. Its those pesky NP jobbies that get in the way.
Whenever the handwaving starts around a discussion relating to a NP hard problem, I find it useful to imagine a Canadian bloke (MHRIP) in a red top, with a ... Scottish accent ... saying:
"Ye cannae break the laws o' physics, Jim". (maffs not fisics, obvs!)
If that is a bit tiresome for the gung-ho AI evangelist, there is also the rather knotty snag that that blasted Austrian geezer Gödel fiddled up: incompleteness.
Its almost as though these bloody clever scientific and that types keep on putting artificial blocks in the way of LLMs laying golden eggs!
I'm quite happy with the "marginal gains" I get with a DGX Spark. It will pay for itself within three months doing stuff on prem and us not sending data to someone else. It will scale.
The sooner people can be broken out of their denial about all this the better, and we can start actually taking it seriously.
Maybe you’re the one who needs breaking out of your cached beliefs.
In my experience modern models are better at all tasks than models from two years ago, especially complex multi-step tasks.
I suspect GPT 5.6 would be even better at it, if given the same sycophantic system prompt and lack of guardrails.
It wasn't "better" it was better at kissing your ass which matches what a lot of people want in a partner.
I think most are actually worth, as agentic harnesses seem to optimize for solving poorly described problems rather than following complex procedures as written. In other words, instruction following maximizing models seem to make worse free-form agents, but they're really all that some domains need.
You can do many more things, when stuff is cheaper, even if the stuff were otherwise unchanged.
For example, every day people teach teenagers how to drive and with only dozens of hours of practice, they are on the road.
Another interesting task would be to take the AI in a robot body into a vegetable garden and teach it to pull weeds. This is another task that lots of children help out with.
That’s not a credible position, but there isn’t anything that I or anyone else can say to someone who simply doesn’t want to believe something.
I don't know if I agree with that but it doesn't seem like an irrational claim and does seem credible to me.
so far there is no end to this progress in sight so it's full steam ahead on this singular domain. once it plateaus you should expect to see the greatest disruptions in human endeavors ever as all the training flops will start flowing to other domains to disrupt and dominate.
If your source is AI layoffs, there were plenty of layoffs with the invention of the horseless carriage, but that doesn't mean it made humans worse off overall.
There are reasons to be skeptical about progress and AI and all that but the 'making things worse for most people' thing you mentioned seems yet to be based in any reality
Is the amount of people making money from AI greater than the amount of people who lose money from AI? Is maximizing prosperity across humanity even a goal, at this point? Am I to be expected to believe, without doubting, that the end goal is the benefit of the many?
I ,for one, have read enough history to know that it's never the proles who end up benefiting.
And nowadays with Hong Kong (and probably soon Taiwan) they are proving they are perfectly happy to destroy economic growth as long as it benefits The Party
That's not what people mean when they say "moving the goalposts". It means that people are adamant that something wasn't important/hard/impressive once the "AI" solves it. And then they come up with another thing that needs to be solved in order to prove it is important/hard/impressive. And once that happens, they do it again. And again. That's what "moving the goalposts" means.
It's also very much not a new phenomenon. It's been happening since the 1980s. As you can see from this quote from GEB by Hofstadter:
> There is a related "Theorem" about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of "real thinking". The ineluctable core of intelligence is always in that next thing which hasn't yet been programmed. This "Theorem" was first proposed to me by Larry Tesler, so I call it Tesler's Theorem: "AI is whatever hasn't been done yet."
I don't see how that's any better.
Actually, I'm really an LLM and this whole thing was a clever bluff.
-Alan Turing (allegedly)
In recent years, I have commonly seen the phrase "you're moving the goalposts" deployed by the "it might be sentient" crowd to shoot down the "it's a stochastic parrot" crowd when the latter respond to a new development with "OK but...". In a well-understood field of inquiry, that would be a clear case of goalpost-moving, in the commonly-understood meaning of the phrase where requirements are retroactively changed in response to them having been met. Thank you OP. 'Artificial Intelligence', and indeed intelligence in general, is very much not a well-understood field of inquiry - in fact we don't even have a common agreement about what 'intelligence' is. We are therefore learning as we go (even after all this time!) but making rapid progress in recent years. When rapid progress is made in a poorly-understood field, then how can our definitions and requirements for success not change? This is arguably one of the most pathological development projects ever - what are the requirements? 'It thinks like a human'? What does that mean? And the answer is we don't know what that means, and we're working it out as we go - moving the goalposts. If we didn't move the goalposts, then by definition we already knew exactly where we were headed at the beginning, and we very clearly did not.
Side note that, in case it's not obvious, none of this detracts from how impressive LLMs are. They're a marvel of the modern age, all the problems notwithstanding. However I reserve the right to stay sceptical about their capabilities.
It's in how they change, not the fact that they change. The skeptics seem to have secret definitions for intelligence, sentience, consciousness, creativity, etc. that amounts to "a thing only humans have". Often that thing is equivalent to a soul. When yesterday's challenge (LLMs don't have X because they can't do Y!) is met, Y changes but X stays the same. This is not the process by which a field matures, it is a rhetorical technique used by skeptics to avoid honestly stating or confronting their internal definitions. That can be revealed by asking the skeptic the following:
"Forget LLMs. What if we made a completely physically accurate simulation of a human being?"
Many say no, that simulated human being still couldn't have (intelligence, consciousness, sentience, creativity, ...). This reveals that there is a necessary metaphysical component to those attributes, at which point any scientific-minded person will leave the debate.
> If we didn't move the goalposts, then by definition we already knew exactly where we were headed at the beginning, and we very clearly did not.
The criticisms are directed toward people who did clearly act like they knew, not the ones who were honest that they did not know.
To me, the goalposts were already defined by the person you were responding to. "The impact of AI is getting undeniable", so, the goalposts are "the impact of AI". Probably something like "the impact of AI is high, or will be soon".
Note that this does not depend on things like AI sentience or defining "intelligence" more rigorously, it just depends on AI impact.
> "The impact of AI is getting undeniable", so, the goalposts are "the impact of AI". Probably something like "the impact of AI is high, or will be soon".
those are not goalposts - that's an incredible vague 'goal'. What are we moving here exactly?
The fundamental argument that I've personally made since the early days of this is that LLMs are not reasoning, in the way that word is commonly understood.
There are lots of reasons why that argument needs to evolve that could certainly appear to be "moving the goalposts", but let's take an example.
A lot of AIs were tripped up by the question "Should I walk or drive 50m to the carwash?" Several folks liked to use that as an example that illustrates that LLMs aren't reasoning, but as the models have been trained on that specific example, it's of course less useful. An AI can mostly nail it now.
So a different example is needed. A new demonstration of how these things fail at basic reasoning a child can do.
Did I move the goalposts? I don't think so. The fundamental argument stays the same. It's not hard to find lots of examples that trip up LLMs, because they are what they are: statistical inference machines. Nothing more and nothing less.
Useful, sure. But also commonly misapplied to areas for which they are inappropriate solutions.
The motte is "AI useful". The bailey is "Singularity is nigh".
(I'm personally still skeptical about this, but I'm being pulled towards accepting it).
"AI is useful" is too low of a bar, and "singularity is nigh" is too high. "AI is on its way to upending society" is about in the middle, and still vastly contentious among laypeople.
But there are people like Ed Zitron, frequently posted and cited here, who disagree even with the former.
Personally I prefer to follow explorers rather than swamp-sitters.
"it isn't clear whether generative AI actually provides much business value at all"
"cannot seem to find a product that people will pay for, in part because the results are so mediocre"
"Last week, we got our first real, definitive glimpse of what’s around that corner that future. And boy, was it underwhelming."
"OpenAI claims that o1 “performs similarly to PhD students on challenging benchmark tasks in physics, chemistry, and biology.” Just not in geography, it seems. Or basic elementary-level English language tests. Or math. Or programming. "
"Worse still, it's kind of hard to explain why anybody should give a shit about o1."
"o1 shows that OpenAI is both desperate and out of ideas."
"the software is not becoming more useful"
Honestly, every other line is quotable in this context.
I agree with the parent that we need to acknowledge that we're at a turning point in history. I lived through some of them (internet, ubiquitous personal computing). But it's somewhat difficult to comprehend the impact of this one for many people.
I do biomedical research at one of the top European research institutions. We're very well-funded, but I can clearly see the gap between us (say, top-100) and top-10. I also realize this gap is going to get so much wider unless we invest heavily in AI access (and I'm not so sure I can sell anything more expensive than $20 Claude subscription to the leadership).
I think people having 6-7 figure SOTA AI budgets will move exponentially faster than those who don't. That makes me worried.
So, for me, it's not a question of recalibrating expectations. We're way past that.
Not long ago many folks were saying AI was the same as the crypto bubble. No real useful technology and only hype.
I think AI is clearly both revolutionary and useful. Revolutionary insofar as the job I do has changed almost completely in a year or so span.
https://garymarcus.substack.com/p/two-critical-updates-re-as...
As always, PR hype. Goalposts have not moved.
Guys, please use critical thinking. The haters don't hate by default, we hate because we're gaslit about this stuff every day and it's annoying. Extraordinary claims require proof, and they're not giving us information that would be essential to knowing if this is actually significant or not.
Remember October 2024 Pelicans [1] ? It's been only less than 2 years.
We don't know what will come in the next 2 years. But the progress doesn't seem to stop for now.
[1] https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/
People are skeptical of the announcement because the room include several PHDs in math and physics. The prompts are not published so we can see how generic the starting prompt is.
If I could take one out of context quote from this whole thread as a response to TFA, it would be this one.
He literally says it's an impressive feat in the second article.
The only way that is PR hype is if you're invoking the insane conspiracy that frontier AI labs are just buying off results that would otherwise be career defining for a mathematician, just for marketing.
The posts you linked are urging caution regarding the exaggerated e/acc-esque lies peddled by people like Musk, not that the models haven't proven themselves as having genuine ability to contribute to research in some areas.
I remember the time when he insisted that diffusion-based image generators trained on Internet scale data will never be able to make an image of a horse riding an astronaut. Today you can generate 4K video of that.
They still do things that I find incredibly annoying and “dumb”. And I still have to clean up messes they make quite often.
But on the whole they are clearly smarter than before. No extraordinary claims needed. I just try to learn how the tool works and how to use it effectively.
i can link you likely dozens of comments from people wrong about this replying to me over the last 5 years
We need to figure out how to restructure the global economy. How does UBI work internationally, if the AI companies are taking revenue in the US? What’s the tax base for it? What does that say about international trade and protectionism? Do countries end up splitting into different trading blocks based on their level of access and legality of AI (I assume some will ban it outright)?.
How does intellectual property work in an AI generated future? What about healthcare advances, who gets to own those?
What about meaning, what about purpose? How do we replace the work ethic that tells us we are our jobs and idleness is immoral? How do you replace “What do you do?” As one of the first questions you ask a new person?
That sort of thing.
And it's sad, really, because I think these two groups would make a great pairing if they could stop arguing against one another for a moment. They'll both be impacted about as much and probably have the same ultimate goals (to lead dignified lives).
But it seems these days everyone is more interested in Kayfabe and feeling like they're in the right than working together, so maybe I should just keep quiet rather than attract the ire of both groups...
I don't know if it is fair to say they're in denial. For my part, I don't expect life to get much better for regular people (especially short term), but that doesn't mean we shouldn't work to try to make it happen.
What a lot of people want to do, and I’m not saying that you’re one of them, is to assume that a positive outcome is impossible and either do nothing or loudly yell that the world is ending. Neither is particularly useful.
Or, as I said above, others just deny that there’s anything to see here and try to get people to move along.
Shane Legg (DeepMind co-founder), one of the more intelligent and thoughtful people you'll find in the industry, could only offer "it's a tough problem - we need to think about it" when recently interviewed by Hannah Fry.
On the surface the most likely outcome for AI allowed to replace jobs is extraordinarily negative, especially since it is a general capability technology, not a specific one where displaced workers can just move to another field. Once AI becomes more capable it will be able to do the vast majority of white collar jobs, including any new ones that may appear as a result of AI. As Shane Legg put it, "if your job can be done remotely, sitting in front of a computer, then it can probably be replaced by AI".
Not only does AI threaten to replace ALL the white collar jobs, but it is rapidly going after blue collar (factory jobs, driving jobs) and pink collar ones (Japanese robotics for elder-care) as well.
If a positive outcome (which doesn't include putting displaced workers on welfare - UBI) is possible, then it sure would be nice to hear it, and the silence from the AI companies, and government for that matter, is deafening.
Eventually UBI will be the norm, and if the living standards of a person on UBI is as good as yours or mine today, that will be an enormous win for everyone. It’s like pensions, once these were only for the elderly poor, now they’re a right for everyone in most developed countries.
It’s also interesting that for most of human history leisure time was the point of life, and only in recent modernity has work come to be the meaning of someone’s existence.
UBI has to be commensurate with production being automated. That’s a big logistical problem, if you think building datacenters is a challenge try bringing about radical abundance, but even so it’s not insurmountable. It just needs to be taken on as project and not seen as an impossibility.
So much of this is not about what is possible so much as what people believe is possible. We can do anything if we try.
I see comments like this tossed around a lot, but what makes you say this? Don't you think its more likely that most people end up in poverty?
why do they spend the money they do on the things they do?
they are not altruists and they never will be.
they could change millions of lives today, but most do not.
pure naivete.
In any case historical trends are really irrelevant to the discussion, which is about the impact of AI, which threatens to take away almost ALL the jobs (this is what Dario's essay is assuming), which has no historical precedent. Some people like to bring up previous waves of job automation such as the industrial revolution, but the difference there is that automation took some jobs but created others. In the case of AI, AI will also be taking the new jobs that it creates.
If AI takes all the jobs - which is what the people like Dario who are creating it assume will happen - then it seems "UBI" is indeed the logical conclusion (other than those able to make a living working for themself, or via self-sufficiency), but this is not going to be utopia where we are all idle rich practicing our hobbies. What it really means is a welfare state, where formerly proud people capable of supporting themself become dependent on government handouts. What comes to mind is the movie "Soylent Green", not utopia.
I'm not sure how anyone imagines this would actually work. Is there any private enterprise left at all, or are all the means of production (AI datacenters and robotic factories) all controlled by the state. Instead of distributing Soylent Green, the state distributes UBI-scrip, essentially food-stamps, that can be exchanged at state stores for provisions?
Seriously, how could this actually work ?
An alternate future, no more optimistic, at least in the short term, but perhaps more realistic, is that in the fairly near future when unemployment and public pain becomes high enough, we reach a tipping point, and the pitchforks come out. Eventually the government concedes that AI is no more conducive to the public good than nuclear weapons, and heavily regulates it, banning the use of AI to replace jobs. This may sound extreme, but surely not a fraction as extreme at the UBI-based welfare state that Dario Amodei is fantasizing about as the best possible outcome (that is compatible with himself becoming enormously wealthy).
What about other countries in the world where people... live in poverty?
> Capitalism allows people to escape poverty by personal effort
Isn't it possible that there is a future where AI makes the average value of human labor (or, "personal effort") plummet? Perhaps capitalism will lose some of its edge against a technology like this.
See "Machines of Loving Grace" by Dario Amodei: https://darioamodei.com/essay/machines-of-loving-grace.
"Massive Economic Abundance: Because AI will exponentially grow the total economic pie, overall resource scarcity will diminish. The fundamental challenge shifts from producing wealth to distributing wealth."
So how do we go from everyone out of work, no income to spend on food, or the goods and services that the AI is producing, to "massive economic abundance"?!
It's like the meme:
Step 1: Create AI
Step 2: AI takes all the jobs
Step 3: ???
Step 4: Profit! (massive economic abundance)
What is step 3?
"Massive Economic Abundance" implies massive increase in produced goods. This implies massive deflation, ceteris paribus. So step 3 could be simply printing money to pay for UBI. Deflation from AI productivity increase and inflation from UBI money printing will cancel out.
For money to work it has to represent some real value, something that has some scarcity to it such as potatoes or hours of human labor. Ultimately it is just a decoupler in a barter system, a universally recognized IOU.
Why would someone give me a car in exchange for UBI-scrip when that UBI-scrip has no inherent scarcity or value and can be produced in infinite supply by the government ?
UBI script will have some value, I am proposing printing enough money just to combat AI productivity-induced deflation, not infinite UBI money.
If you are correct, I expect corporations to reap massive profits while most Americans try to find a way to survive in a world where they are obsolete.
> In the near term handling the transition. Jobs will be lost, careers ended, people won’t be able to reskill quickly enough. At the same time AI is an enormous opportunity to uplift living standards, but nobody has the logistics of this figured out.
> We need to figure out how to restructure the global economy. How does UBI work internationally, if the AI companies are taking revenue in the US? What’s the tax base for it? What does that say about international trade and protectionism? Do countries end up splitting into different trading blocks based on their level of access and legality of AI (I assume some will ban it outright)?.
UBI in the United States is never going to happen in time. If it happens at all. We don’t even get universal healthcare. I think people who think AI will be a net positive for humanity are also in some sort of denial.
In a different US political climate I would entertain it. If these frontier labs weren’t so clearly going after the money, I would entertain it.
LLMs are clearly a step up for capitalists so I just can’t see any inclusion of LLMs move towards more progressive ideologies.
> How do we replace the work ethic that tells us we are our jobs and idleness is immoral?
For many people it has nothing to do with morality, it's hardwired into their instincts. They want to work, and they will work.
For a lot of us who are not excited about this future it's that no one is trying to answer all the questions you laid out. Instead we have the disgusting people at the helm purposefully spreading doomerism and saying, "We'll figure it out." I think it's pretty problematic (to say the least) to care more about technological advancement than how that advancement is actually shaping up to effect people in the short term. But I know many people don't care, especially those who believe they won't be among the affected.
My own $0.02 on the economics piece - every country should have a sovereign wealth fund. Governments should block market access from automated[0] companies until those companies provide equity contributions to the wealth fund for that country. This aligns regulator and corporate interests. Dividends flow into the sovereign wealth funds and then can be allocated locally from there - UBI, job programs, etc. Let different jurisdictions explore different ways to structure a post-labor society.
On the broader social front - I think a lot of lack of meaning discussion boils down to the overemphasis we have on your job as your self-worth. We need to realign our societal expectations - and people need to spend more time with their families.
[0] for this to work, I think we would need well accepted metrics for 'how automated' a company is - and that probably needs a 3rd party auditing industry.
Is there any government that has gotten socialism correct for its citizens? I'd point to UAE/Qatar if they didn't depend on human servitude and inequality.
I'm only suggesting (eventual) negotiations between countries and automated corporations - to operate here, you contribute equity.
The only way to win is to wield the AI.
Humanity survives (but we reading this probably don't), the AI treats the living humans like the Emperor's favorite pets (probably a pretty good life), and then the AI does whatever else it deems important.
Bad news for you -- there's a 100% chance we all die. Sorry to be the one to tell you.
And you accuse the 'other side' of 'suicidal apathy'??
You should put down the AI and do some self-reflection on how you came to hold these views.
I've been in the field for almost 2 decades, and actively thinking about AI risk for over 15 years.
I've always held the controversial opinion that there may come a time where we might unfortunately have to consider using force to protect our civilisation from the threat of ASI, but I've always reframed from openly discussing that because I'm so personally against any use of force and there's always been time for more peaceful options.
Several months ago I changed my tone on AI risk to stop worrying about the optics of what I'm saying because it's too late in the day for me not to speak plainly at this point. Similarly over the last week I've decided I can no longer reframe from advocating for the use of force (if necessary) to stop rouge actors from playing Russian roulette with civilisation.
Please understand I don't want anyone to be harmed. Perhaps you don't agree with my suggestion, but my stance ultimately comes from a position of harm reduction.
> preemptively against a 20% number you just made up?
It's a probability estimate. Happy to expand in detail on why 20% specifically, but first you accept I can't tell you what's going to happen in the future with certainty right? To some degree any prediction you might have about the future will be "made up".
There are dozens of escalating, yet fundamentally less destructive means to prevent a data centers from continuing to operate (if that's the goal). Not starting at those more modest interventions is political malpractice and in opposition to your harm reduction narrative.
Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
If you're referring to the "nuke the data centers" line in my bio, that's tongue in cheek. It's an expression of my sentiment towards AI at the extreme.
To be clear, I don't think any country should seriously be considering dropping nukes on data centers. Targeted strikes on data centers operated by rogue private companies or nation states is acceptable to me – assuming pre-warning of intent and appropriate regulatory foreshadowing.
I suspect we agree this is reasonable?
> Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
> Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
No, they're not based on unbounded exponential growth curves at all... I'm actually somewhat skeptical of claims that there could exist an intelligence that far exceeds our own. My assumption is really just that AI will be far faster, far more knowledgable, far more persistent and marginally more generally intelligent than the average human. The first three are already true, the forth we're getting to.
We're on an exponential curve right now and I suspect the rate of progress will accelerate some more as a result of RSI. This will likely continue for 2-3 years (if we have that long) before starting to flatten out.
As for my doom prediction, my median concerns are around around the destabilising nature of ASI.
If it can be controlled then it will concentrate power like never before, and I believe this will destabilise civilisation. I can go into more detail here if you wish.
However, I'd still put these destabilisation risks at under 50% in total. My concern more broadly is the majority of ASI outcomes are doom scenarios. It's extremely hard to imagine a world in which you have a magic box that can grant any wish that intelligence can grant, but which we have a functional civilisation. You have to basically assume all potentially dangerous wishes won't be granted and probability the only feasible way to do that is restrict who can make the wishes – but then you have the concentrate of power risk.
I largely agree with the premise "If Anyone Builds It, Everyone Dies", so we must try to avoid building it. I think we can build ASI safely, but we're clearly not on the path right now and I suspect it might take decades (or more) to figure out how we can build ASI with an extremely high probability of a good outcome.
> Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
I do this. I've been writing endless comments more recently urging people worrying about their jobs/careers to stop and just spend the next few years with those they love.
I don't think we realistically have a chance of diverting from this path at this point. I have made peace with this but it makes me very sad and haunts my dreams almost every night.
We already live in a reality with unprecedented concentration of wealth and power in the hands of the very few. This process has produced horrific violence and ended civilization for tens of millions.
The actual impact of AI, so far, has been redistributive upwards (solely through speculation, not via productivity or displacement).
This kind of problem is a standard human social ill. Its solution is social too.
Redistributive economic and political action can equalize power and wealth disparity. AI is incidental in this story.
Your understanding of AI as an isolated perfected intellectual djinn lacks a consideration of the existing and ever-changing labour and social relations that give AI responses meaning in the first place.
It is a compressed, multi dimensional manifold containing the structure of relations between objects. Responses are a reification of a point in that space.
Navigating the space can be interesting, but it is a poor substitute for the relations themselves.
Very few people yearn for the endless, frictionless solipsism silicon valley promises (AI + slave robots), and those that do are overrepresented in your social circles. The books they write are a study in psychological projection and limited political education.
> I've been writing endless comments more recently urging people worrying about their jobs/careers to stop and just spend the next few years with those they love.
This is good advice generally. But it isn't organizing. Join something, meet up, drink bad coffee, listen, take notes, advocate to fix just one bad thing in your community, feel your impact reverberate through the world, and then notice how much easier it is to shake off despair.
If you don't have a community, then no wonder everything feels like its ending.
Does it bother you that the people who are publicly cocksure that P(doom) is moments away are the same people that have profited most handsomely from that pronouncement?
That the 'humanists' that want to do 'altruism' for 'potential future humans' and are the same people that commit fraud and theft at a civilizational scale, then sell this 'intelligence' to any child-incinerating militaries with spare cash?
It's not wrong to want to do good, but if a system that is branded 'do (the most) good' commits great evils, you are morally and intellectually obligated to step back and reconsider how you are spending your time.
Also, I asked a chicken and a feral rock dove what it's like to be not be 'apex' and they burbled at me and kept eating millet and sunflower seeds.
Would you like me to follow up with them? I'm not sure what point you expected them to make.
You hallucinated the "preemptive nuclear war". He didn't say anything about nukes. That's your own invention.
> Does it bother you that the people who are publicly cocksure that P(doom) is moments away
20% is not "cocksure". The "moments" is again an exaggeration.
Every problem is a search problem. Nuke the data centers.
P(doom) = 98.9% (Aug-2026) P(doom) = 98.2% (July-2026) P(doom) = 98.2% (Jun-2026) P(doom) = 98.5% (May-2026) P(doom) = 98.8% (mid-April-2026) P(doom) = 98.7% (April-2026) P(doom) = 98.7% (March-2026) P(doom) = 98.5% (mid-Feb-2026) P(doom) = 97% (Feb-2026) P(doom) = 94% (Jan-2026) P(doom) = 93% (Dec-2025) P(doom) = 95% (July-2025)
It's was an expression of my sentiment, not a policy position I'd support. I wrongly assumed that was obvious, but some people are making bad-faith assumptions about me and my sanity.
Not that it should need to be said on a comment thread where I am express concern about civilisation in a post-ASI world, but I obviously don't want to see a nuclear war.
Powerful word, `if`. "You're not only wrong you're a fulminating psychopath" is a perfectly valid response to getting it wrong like a fulminating psychopath.
What can be asserted without evidence can also be dismissed without evidence.
AI is a math & computer science problem. If AI can do advanced maths and computer science research, then it can begin to suggest useful algorithmic optimisations.
Right now I'm sure the vast majority of these will be junk, but occasionally, even with current limitations, they might occasionally stumble on something.
It's not really whether RSI is or isn't possible, it's really just whether it's the most efficient way for labs to improve their models today given they have limited compute to run AI-generated experiments and access to very intelligent humans who might have a better hit/miss ratio.
Do you disagree with anything I'm saying here? Do you not think current AIs can suggest algorithmic improvements or something?
It's fine, and I'm very used to this, but if you want to have a dialogue I'd be very happy to. I'm a very reasonable and sane person outside of apparently holding some controversial views on AI risk =)
I'd love to know why you think what I said was insane?
Now you are again postulating that there would be no more extreme progress in the near future. That's actually more "insane".
I’m not a mathematician so I have zero clue what “ New upper bounds on sphere-packing density down to the Cohn–Elkies thresholds” means.
not sure how many will get this reference but "AI" for science and math is like super-shoes for runners
at first we are blown away by the impossible improvements including sub-2-hour realworld marathon and every other PR/CR/WR is dialed down
but then the improvements slow and reach a stall point because of the limit of technology and the source of the achievement
ie. sub-2-hour marathon yes, sub-1-hour never happening (rollerblade inline-skate record is 1-hour marathon)
The fact we see a lift is not the same as evidence that the lift is unbounded.
The lift being finite is supported by the fact improvements have come at the edges: improvements from human feedback, improvements in harnesses, improvements on model compatibility with harnesses, improvements in inference efficiency with new architectures, etc. If we were just training better models from scratch that would be one thing, but we are just making better use of a tool we've developed.
As a programmer, I am mostly interested in whether my role is sustainable long-term and whether the models will get better. I don't feel in jeopardy yet, but two more years like this and the calculus of hiring software engineers could shift even further. QAs are already overwhelmed with work
but with super-shoes more and more runners are qualifying for boston marathon and even olympic trials marathon where it would have been impossible for them previously
and that's what "AI" currently does, it allows average people to immediately "pick the brain" of every expert in every field, in every scientific paper, without previously reading a single other google result, something that would have been impossible for them previously (super-shoes for the brain? too far?)
but "AI" isn't creating new knowledge, it's just stitching together existing knowledge from patterns that would have taken years by human hand if even possible at all, it's going to "hit the wall" eventually (in its current form)
basically everything Benjamin Franklin did was trial and error because no-one understood what electricity was in the slightest
almost everything Edison did was trial and error too, he had his lab try thousands of materials for his long lasting lightbulb filament
even the most advanced "AI" today is just machine-learning going through everything already known trying to piece together previously discovered facts, admittedly at levels and detail impossible by human hands
but that means there are limits and it's not really "AI"
I have seen it produce tentative genetics for experiments, just like a scientist does. Then the data comes in and it can evaluate the data from the experiment just as well. One just had to give it a lab budget.
I heard that Gary Kasparov was impacted by AI chess, but at least he still seems to have a job, so don't give up.
My point being that AI math is a narrow skill just like AI chess and implies nothing about generality (AGI).
Sarcasm begats sarcasm.
It is true there hasn't been a reliable computational approach to solving these problems before. But do these proofs contribute new ideas to the mathematical corpus, or are they simply an effective method to exhaustively search the literature for the right combination of existing tools to apply to the problem?
Essentially, did these problems seem like they had an intuitive answer and were feasible to prove before, just not high enough value targets for an expert to invest time into? Or were they fundamentally difficult prior to this point and it appears that AI has done something more than just throw the problem into a big solver.
The sofic groups question was the outstanding question about sofic groups. Almost everyone thought that non-sofic groups existed, and there were plausible candidates, but proving a group was non-sofic was out of reach. Now that we know how to do it once, we can probably do it a lot more.
The Connes rigidity conjecture I think people thought was false, but it was a provocative claim to make. The significance of conjectures is frequently not that the answer to the question is "yes", but that we don't know how to answer the question. And now, apparently, we do.
a colleague was telling me that the base idea for proving that something is not sofic already appeared in the literature around 2019 or so (this is the "expanders graphs" that are mentioned in OpenAI s paper. no one had managed to find a concrete example though. this doesn't make the result less impressive in any case.
The general consensus of developers is that AI can only do the work of a strong 'junior'. Yet as soon as we are presented with pure mathematical results, people seem incredibly ready to accept that AI can do more than what a strong student could achieve.
If it works better here than for programming, then I would guess it's because you can give it a very precise prompt, so you either solve the problem or you don't. If you read the prompts people have shared for problems like this, then the instructions are basically "Solve this problem. Don't give up early. Don't solve a similar problem."
But the wording of the result makes it sound like we don't know what the lowest possible complexity bound might be. So, prior to this result did we think there couldn't be a lower possible bound? Or did the arithmetic circuit community think there were lower possible bounds but didn't see it as a high value target for experts to tackle (maybe a problem that was instead regularly given to students to study).
For example, despite our best efforts, the state of the art lower bounds on time complexity of algorithms for solving 3SAT is O(n). In contrast, our best algorithms for the task run in time roughly O(2^n). That’s an exponential gap. This is despite decades of trying to find lower bounds.
Your worry.... is because they used the word advanced? For marketing? The word is used very appropriately here. There were PhD's who spent a big part of their career tackling these problems.
I'm trying to understand if these specific problems were the kinds of problems that would have justified an expert investing weeks or months to solve. Or if they were the kinds of problems that would normally have been given to students to investigate.
Mundane incremental research is cobbled from existing citations that already appear nearby in the record.
Basically, innovative research is a measure of bridging thought and domains that were previously not bridged. It's quite concrete as a measure in the citation record.
So we can know pretty conclusively.
Puja Ohlhaver gave a talk on this[1], and ran some experiments (that I had the pleasure to support on)
Also on HN front page today: AI's debt binge can't last, hidden borrowing reaches $1.65T (fortune.com)
Similarly, if they are spending millions of dollars in inference just churning on thousands of problems and these are the 10 solutions they came up with, that would be less impressive than if they chose these 10 problems specifically and were able to come up with solutions.
but it seems less likely to me than before that the types of math/science discoveries will explicitly unlock better software performance. in some sense this fits our intuitions. when top tech companies use math PhD type employees, they have them stop doing pure math research and instead focus on software engineering. these people are often very good at software engineering but not due to recent discoveries in academic mathematics, it's due to their general intelligence. to me, this is evidence that the models are getting better but does not make me think we are on the cusp of a foom style fast takeoff enabled by revolutions in frontier math (i also posted this on twitter @mlipman13)
If I had to create a tagline to describe my opinions about AI in a single sentence, that’d be it.
It’s possible to both hate AI and be impressed by it at the same time. Lying to ourselves about its capabilities does us no good. It’s emotionally difficult to do, but people need to come to grips with what’s happening and shake themselves out of a state of denial.
The AI results are clearly impressive. But these sorts of things are also in the ballpark of what human effort could solve given enough attention and time. Though it is hard to say.
Like, these would be best-paper awards at many top CS conferences.
You seem to overlook a simpler barrier. To make these advances, they have to be possible. A 15% improvement in GPU kernels doesn't evidence that significantly more improvement has been left on the table.
Incredible?
> open ai announced like 15% improvement by fixing gpu kernel issue
That is... ordinary software optimization.
Edit: also here’s a opencl 30% compute perf increase documented here : https://m.hexus.net/tech/news/graphics/74425-haswell-systems... that i just googled for
> we are seeing frontier level math breakthroughs (ie performance that would put it in the top 100 or 1000 mathematicians in the world if it were a human, meaning top .00001% or 800/8B)
i think you have misunderstanding of what mathematicians do
They get to make cool 3D plot visualizations of functions so obscure to me that they’re named after someone who is still alive - and/or get to work on cryptography for the NSA - I think?
> You are an expert in the field of mathematics, with decades of experience. You are a reviewer of proofs, etc etc.etc.
It also links to a paper written by an LLM where the model "reconstructs how the proof came together" based on the unpublished reasoning traces: https://cdn.openai.com/pdf/reasoning-walkthroughs.pdf
I wish they'd publish the prompts though!
I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications) that the lean proofs are checked against. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.
This is an extraction from that of the actual theorem statement (39 lines): https://github.com/openai/ten-proofs/blob/94bc0feb6a9ff12c7d...
As long as you are not missing important information, how you word the prompt does not have any effect.
check here : 1. high dimensional sphere packing https://muchmirul.github.io/conjectures/sphere-packing/
2. multicolor ramsey number https://muchmirul.github.io/conjectures/multicolor-ramsey
So I would like to counter your cynicism with a “YMMV” depending on who you work for.
What does "support" mean in this context?
Remember, this is the propaganda arm of a VC. Proof is a hindrance to the hype that drives value.
Like cool my lung xray only took minutes to determine if I have a lesion instead of a week or a few days, but I still have cancer.
More importantly, if you can screen for cancer in a way that takes minutes instead of a week, imagine how accessible this technology will become.
- travel and restaurant recommendations. my last few outings have been entirely LLM-advised and they turned out excellent. LLMs seem to have ingested every single Google review, photo, and menu of every business on Earth and can answer very nuanced questions like "is the garlic chicken at <restaurant, city> garnished with coriander?"
- fitness, nutrition, accounting, therapy, medical, legal, immigration advice (sure it's not a real professional but you know what, it's pretty fucking close, and any capability gap is made up by having perfect two-way communication which you don't get when talking with a human)
- coding (work, side projects, personal tools, documentation & pricing questions, "review this code", etc).
- I start reading most articles with the prompt "Summarize this article: <url>". I just started a non-fiction book by pasting into Claude: "There are 12 chapters in the book <book-name>. Can you give me a 2 sentence synopsis of each chapter?". It reduces the "activation energy" hump and screens if it's worth reading at all.
- I use the LLM in my Tesla for on-the-fly advice for parking and other things. You can simply ask "what's the best Boba place around here?" and it will give you a decent recommendation. You can also follow up with "does this place have ample parking?".
- I use the LLM in YouTube to summarize videos and ask specific questions and/or get timestamps to the parts I care about.
If your critical thinking skills are strong then LLM is a literal superpower.
Where can I find this one?
I have a janky pipeline built on top of yt-dlp and I've been wondering how many more years until they make it so I don't have to do that anymore.
(Though sometimes I'll ask Gemini — the YouTube integration is the only reason I use it these days.)
2. Do LLMs reduce the loneliness epidemic?
3. Do they reduce population aging in almost all counties around the world?
4. Do they reduce political polarization?
5. Do they bolster democracies?
6. Do they decelerate climate change and general environmental destruction?
7. Do they accelerate sustainability and the circular economy (not circular financing!)?
8. Do they reduce the workweek and give people more free time for family and hobbies?
Etc, etc.
I would hold off on calling anything a "superpower" unless it solves the hard problems in life. Heck, computers and even the internet barely score better than LLMs when measured against the important things in life.
The QoL improvement from LLM is somewhere in the ballpark of going from dumb phones to smart phones for an average person (able-bodied, local to the area etc). It's nowhere near Haber-process or penicillin.
Well maybe not the LLMs, but AI and robotics have a lot of potential here.
It's still the early days though. Try again in 15 years!
Sure. Often to zero.
https://www.reuters.com/world/us/americans-fear-ai-permanent...
But I don't think the argument needs to be "AI is like gambling". The argument only needs to be "humans often behave irrationally and even self-destructively".
Perhaps you can ask Claude to explain it to you.
I want to know:
1. How many total problems were given to the model, and what percent were left unsolved at what cost before giving up? 2. How many attempts did you give the model at solving these problems? 3. How expensive was the harness, e.g. did the model have access to a job cluster?
https://x.com/polynoamial/status/2083478171975082334
As a complete guess, it seems like they tested hundreds to thousands of problems with a relatively low per-problem budget
--
The linked tweet from Noam Brown at OpenAI reads:
> And yes we did try other major problems without success. Sadly no Millennium Prize problems (yet).
> But also, we didn’t spend a lot on each problem. It’s possible to push test-time compute much further.
It's not just about requiring to disclose AI use. AI-powered mathematics is a completely valid discipline that doesn't need to be shy, but it should develop its own publication culture.
Pure math is relatively outside my domain, so I find it difficult to grok the exact relevance of the various published discoveries beyond that they are not insignificant, and LLM competence is expanding quite steadily across the field. If this trend continues to the point of LLMs being able to competently expand pure math, it seems somewhat predictable to expect there to be a number of people aiming to find ways to try to keep human mathematicians in the loop.
I've no idea what I think about this one way or the other, beyond that it's certainly a phenomena and one that's going to drive motivated reasoning that may not be entirely sound.
My perspective is more like a FOSS philosophy for math. Even if a closed version has the same immediate effect, it's just better for everyone if everyone can look under the hood and tinker with it.
I would've thought pretty much the exact opposite. "Prompt engineering" was somewhat important in 2023/2024 when the models were much weaker, it doesn't seem at all necessary anymore (unless just "clearly stating your requirements" counts as prompt engineering). Most of the discussion I've seen seems consistent with this?
Methods are only really necessary for results at a meta level, about the design amd evaluation of AI math systems.
shouldnt the paper be the math of the argument? the reproduction is reading the following the proof
Another case I want to highlight is writing GPU kernels as illustrated by the following example: Say I want to generate random number with Normal (0, 1) distribution. Often times the AI written kernel will just generate the number 0. The tests often fail to catch these errors.
Even if the cost was $1 mil for these 10 problems, that's maybe 10-20 math researchers for a year.
Do you really think that if you paid that to humans, they will deliver the same results?
And frankly these "concerns" ignore reality. In any research phd course you're actively told to bite off something small and likely to be provable so that you can prove it (and publish it). Openai telling its computer to do that is no different that your phd advisor telling you that.
The post that started this sub-thread asked:
> 1. How many total problems were given to the model, and what percent were left unsolved at what cost before giving up? 2. How many attempts did you give the model at solving these problems? 3. How expensive was the harness, e.g. did the model have access to a job cluster?
I think it's an extremely relevant question to ask, because it helps us better understand the current state of AI being able to handle math, for exactly the reasons I outlined. I was arguing against the idea this is just a reactionary anti-AI kind of question to ask. It's not! You can be very impressed by what AI is capable of in math (I am) and still think those are really interesting things for OpenAI to disclose (I do).
OpenAI specifically called out a $2000 per problem average, which implies something that's probably not true ("if you throw $2k at us we'll solve an open problem for you"). It would be cool to know what the actual number is.
If these 10 problems were solved by humans, it would be pretty impressive, even if it took a large number of researchers! Yet when AI does it, HN commenters suddenly feel the urge to play accountant.
But that's the start of math research, not the end.
The point is to get practice and experience doing research.
Did ChatGPT learn anything from these proofs, that it can build on?
Part of what's annoying people is that ChatGPT is churning though problems that are meant to be motivating. They are problems that aren't worth the effort of human professionals (usually because they are incredibly computation-hevy, so better suited for a computer than a human), so they are good for students to work on.
I know it's more exciting to say "AI disproved a longstanding conjecture" vs to say "it did so AND it took several PhD specialists in the field this many attempts to even produce a prompt that got the model spitting out something useful under some configurations, and many iterations to optimize the configurations, and the prompt itself, and many trials with that configuration to solve the problem. All told we spent more than a typical math academic can hope make in their career."
By not being transparent, they invite skepticism and cynical takes, like maybe it's just that tempered and qualified claims are an existential threat to companies that are fully subsidized by the hype train?
I don't know. Either way, it seems like it would be easy to address these, so why should they not do it?
To be clear, even if that tempered version is close to reality, it doesn't make the models not useful! It just forces a certain calibration of expectations
I say this btw as someone who uses these things extensively, including to disprove an old conjecture my advisor and I were stuck on recently. I know they are powerful and that everything is different now because of them. Let's be sober when discussing them though
That's not normally how people act when they're confident in their product
The cost of running a model is not only $/token, but the salaries of the people managing/orchestrating the models, deciding what theorems to try, etc. Once we factor that in, how much are we really paying per theorem?
The other factor is the subjective component of the value of a theorem. Not all theorems are created equal, and the only way to really measure the value is to ask professional mathematicians for their opinion, or publish the results and look at citations over months/years.
Once we have both of these nailed down, then we can start to do the cost/benefit analysis. To be fair, we should actually compare three groups: human experts, hybrid agent/human expert teams, and fully autonomous agents.
OK I’ll grant that it’s not your obligation to be my search function (despite you making the wild assertion in the first place), so instead can you just point us to the latest grad student solved problem of this level that you know of?
It's a marketing post from a huge company. Only the naive would view it uncritically without assuming it's been written carefully to present the results in the best possible light while skirting the boundaries of outright lying.
Imagine 2 years from now: "yes, GPT solved the Riemann Hypothesis, but cmon, it's just a marketing stunt to hype their stuff, it was probably Terence Tao doing the work but he's so obsessed with hyping AI that he doesn't want to take credit"
We're saying look critically at the claims for how it was done, that it only cost $2000, etc. it would be extremely easy to run 100 sessions that failed, each costing ~$2000, and then just publishing an article about the one that succeeded, for example.
This goes double since it's an internal secret model (Astra) so nobody else can verify the results.
More generally, do you expect that there's some capability threshold where people will no longer study or analyze AI model outputs, and instead just sit there slack jawed saying "so cool!" every time OpenAI announces novel ones? I don't really understand why that would be or why someone would want that. If you're interested in the pure experience of a complex machine outputting satisfying results, I'd recommend getting into sports cars.
Look at their recent claims about their model "escaping" - there was literally a Guardian article calling them out for being hyperbolic! Again, it wasn't that they lied, their marketing department is too savvy for that. They just present it in way that's, well, marketing.
As for the actual result, I'll look for secondary posts by actual mathematicians and draw my conclusions there, not from this marketing blog post about results from a secret model.
Company X does not make money from proving theorems but does make money from selling you a service which supposedly proves theorems. Company X then proves some theorems and explicitly calls out they were very cheap to prove using its service.
And you think you're actually clever for taking these facts at face value? Interesting.
i would classify you as a flat-earther if that happens.
brother like 3 people have pointed out what they're skeptcal of is cost not LLMs - at this point you're willfully misconstruing what people are saying to you just to get a kick out of repeating your same tired strawman.
If OpenAI solved Reimanns hypothesis and the first comment is says something about lack of transparency and marketing, i would say it’s ignorant.
If OpenAI claimed the conjecture to be true but provided no details about the proof then the first comment should absolutely be about lack of transparency.
do you really imagine a scenario where OpenAI would claim to solve it and not give details about the proof? how is this even possible? why would anyone believe them?
I don't think that comparison to p-hacking is fair. I mean not reporting price of all run is nothing like committing scientific fraud and fake results.
Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.
A couple small ones that I've seen (example here [0]), but not anything of the magnitude that OpenAI and Anthropic have put out. Likely just related to token limits.
> Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.
I think their output has reached a level that precludes this possibility, but I of course don't have any hard proof.
[0]: https://www.reddit.com/r/math/comments/1uxj3cy/after_openais...
I have no affiliation whatsoever with any AI company, nor any formal education outside high school, for what it's worth. Simply being curious and persistent can get you quite far in my anecdotal experience.
https://arxiv.org/html/2605.22763v1
> Our most capable agent autonomously resolved 9 of 353 open Erdős problems at the per-problem cost of a few hundred dollars, proved 44/492 OEIS conjectures
(Well to be more precise we’re going hybrid because of unknown SCAs.)
Anyway, unless you mean conjectures that humans have already posited, or ones that are particularly famous, that list is much shorter, but also contains conjectures that I’m not convinced can be solved before the heat death of the universe using all available compute power. P != NP is a conjecture, for example. Also a lot of prime number conjectures that are extremely computationally expensive.
Now I understand that it’s mostly the super stars benefitting from the increased attention. Folks who are less established don’t share in that glory. But on the other hand it seems like an exciting time to go even deeper for in various specialties of math by deciding where to focus these powerful tools. For every conjecture defeated some seven or eight new ideas open up. Our path through that combination will be set by creative and curious human mathematicians.
[edit: deleted a distracting comparison to Chess]
Which is less interesting work. And you probably need to do the hard grunt work by hand first to develop the skills and intuition to be able to verify an AI-generated result. So you can’t outsource everything to AI without loss of skill.
If you translate that to math, then all you get is math competitions, not math as a career. Of course the translation isn't nearly exact ... there's a lot more room for professional mathematicians because the math space is far more vast than the chess space and can't generally be cranked out mechanically (we have proof).
P.S. The response is nonsense ... I explained exactly why it's awful (others have too) and the response doesn't in any way refute the explanation ... rather it offers up a ridiculous strawman.
these LLMs are great are generating arguments but they don't ask questions, we will need mathematicians to shepherd them into more discoveries
i really want to see open weight models crack some breakthroughs
Was this different before chess computers were invented?
Thinking of this a little bit with the perspective of every new proof as a burden, dumped for review by actual mathematicians.
The most disappointed folks will simply drop out, but the enthusiastic ones will keep going and with luck make up for the ones who decided to quit. Chess and go certainly went this way.
> "Already, there are very few mathematicians qualified to verify OpenAI’s new results. As progress continues, that number will approach zero."
Not that I'm good enough at math to have any uniquely formed opinion, but after reading commentary from people who are, my impression is that these new results are bamboozling the humans due to using tools from so many disparate areas.
At least, we can say that there isn't a single human who is smart enough to understand all ten proofs, even if there is a collective sense in which all proofs are understood.
Maybe good AI paper writing is further away than I thought...
I'd honestly rather they just automate every job at that point.
For example, will we see advances in material science, medical cures, etc?
Would love to read about some examples of practical impact.
OpenAI and Anthropic are too greedy for cash to do anything of the sort.
I don't expect this current economic cycle to bring anything else that will directly greatly improve the life of the average person on the planet, more than it hurts it.
AI has just gotten to the intelligence that it can make clever counterexamples to mathematical conjectures, but the frontier isn't quite smart enough that it can make novel contributions to mathematics. We are really close though. Only a matter of months away.
VP and VNP are closely related algebraic analogues of P and NP. Here the paper proves new lower bounds for computing the permanent, a VNP-complete polynomial, in particular models of arithmetic computation: roughly (n^2\log\log n) arithmetic gates for unrestricted division-free circuits, and (n^4/\log n) size for the more restrictive formula model. These are still polynomial bounds, so they do not separate VP from VNP. But lower bounds on the resources needed to compute explicit functions are exactly what would ultimately be required for such a separation, and meaningful lower bounds of this kind are exceptionally rare.
Training the model is going to be amortized over other uses.
Say that it turned out that the total cost of the proof of the Erdős unit-distance conjecture was $50 million.
Then the question really becomes: yes, these models are capable of proving important mathematical results, but at a very high cost. Is it worth it?
If a mathematician applied for a research grant of $50M USD for proving the same thing, they would have been laughed out of the bank.
What's more is that when you have a research grant, you train PhDs and postdocs, you hire new staff, and you disseminate. That is, you get much more value for the money spent.
I'm just curious what the cost is.
Hours needed for prompt + Hours needed to check result + API costs.
You don't say "well let's add together the total yearly compensation of all the engineers and mathematicians at OpenAI that were involved" and throw that into the total cost. That's simply nonsense accounting.
The actual comparison you are making is some university researcher weighing between getting a grad student (several tens of thousands of dollars) vs typing up a prompt and sending a request to OpenAI for inference (as mentioned in the article, around $2000 in API and maybe a few hours for the prompt and harness).
> We helped prepare the manuscripts and formalize the proofs in Lean, and we take responsibility for their correctness
Offering to take responsibility for the correctness of a proof written in Lean feels like volunteering to be the fall guy in case someone finds a flaw in basic arithmetic, no?
https://leanprover.zulipchat.com/#narrow/channel/270676-lean...
As I currently understand it, all we know is that:
- a mathematician produced a Lean-verified counterexample to the Collatz conjecture, demonstrating a bug in the kernel
- he claims that LLMs were involved somehow but pointedly refuses to specify how
- he admits that he knew about the bug before publishing the counterexample to his repository.
Perhaps not a joke (although it sure seems to me like they discovered a bug and thought falsely disproving the Collatz conjecture would be a flashy way to announce it), but at best extremely sensationalized by the above description. If you have additional context I would be happy to hear it!
I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications), that the lean proofs are checked against, indeed correctly encode the real theorems. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.
But it might be not good for human brains, because we trained our brains with these problems and our brain optimized search space in some ways, and yes, we also couldn't solve some these problems.
Now imagine someone gets stuck with a problem which could become its own theory, but they will solve it with LLMs and move on to solve their primary problem, because they don't realize how other problem was a big deal. If theory is not formalized, then it won't contribute to the search space for other person, solving different problem.
All in all:
* people's brain will be shaped differently
* we will lose search space optimizations in our brains
* we lose new theory contributions, which increases the search space to help solve other problems
Edit: Oh, are you suggesting they just use it to privately improve their models? I imagine a few more correct proofs would have a very marginal benefit, if any. Also, they'll probably just get extracted, meaning it still gets out but OpenAI doesn't get to fancily announce it themselves.
Look at this two threads.
It is indeed true that all models are, at their core, predictors of what occurs next in a sequence. But I think it's worth exploring the implication of what that means. Because when fed tiny pieces of information for a few tasks at a small scale, this results in something that sorta, kinda works. Or, works surprisingly well.
But when scaled... When the amount of information starts approaching the sum of all human knowledge, the tasks start approaching all useful applications of that human knowledge, and the fidelity of the predictor approaches incomprehensible sizes, the starts encodes / becomes (I'd argue it becomes) something that can model all human knowledge.
It feels wrong to say that, but let me explain, what is the best way to predict the behavior of a ball constrained in two directions that bounces with initial vertical velocity v(y) (y is up / down axis) and horizontal velocity v(x) (x is side by side in 1d) ?
If we purely look at it via a graph, it's by modelling the function of acceleration under earth's gravity.
If only a few points are given to you for this and you can't make something really sophisticated, then you'll make something that's rough that kinda sorta works and then call it a day.
But... if the number of points keeps increasing in number, precision and accuracy as well as the number of examples (assumed that data about air pressure, velocity and all other factors is included alongside these points), the fidelity with which you can replay / tweak the function keeps improving, and the number of times you can iterate keeps increasing, you'll eventually create a function that models that process so well that it intrinsically contains a good enough model of the deformation of the ball (provided the dataset contains information about elasticity of the ball's material, its dimensions and mass etc..), the nearly negligible (under normal conditions) effects of the ambient environment (provided there's diversity in the number of environments supplied), the oblateness of the Earth and minute changes in the gravitational field (the length of a seconds pendulum varies depending on where the experiment happens. It's presumed that all of the prior set of experiments were repeated across the Earth and the subtle, but real deviations were faithfully recorded)... and so much more.
A machine trained on the above with a large number of parameters, measures to prevent "laziness" and enough reps for high fidelity across a large enough dataset would start to approach a simulation of the ball falling. Because to predict what happens next in the sequence, you must model what's occurring in the sequence.
Now imagine doing that for other tangible and intangible things in this world. For all of human knowledge across all fields of endeavor. All experiences. No matter how noble, ignoble, notable or ignorable. But putting all of it into the soup that's this machine. Then at larger and larger scales, you eventually start encountering "good enough" models (in modelling the falling ball sense) for even the most hard to quantify / qualify things like grief and joy. At some point, by simply trying to predict what it has been taught ought to be the next part of the sequence in say... human interaction, it starts to make a model of something that hews ever closer to a full fidelity theory of mind.
Is there evidence for this? Kind of, yes. There are early indications that as machines are trained for an ever larger number of tasks at larger and larger scales, their internal representations converge. It's called the Platonic Representation Hypothesis. Overview and paper here, https://phillipi.github.io/prh/
It is my opinion that these machines are displaying a new form of intelligence that human beings haven't quite encountered before. They are the sum of all human knowledge made manifest and given voice by processes that nudge (bit-by-bit) what kind of step it ought to predict for the next part of whatever sequence it displays.
In my mind this means that, of course, these models can create new knowledge. This strains the analogy, but with the sum of all human mathematics within them, they can "reason" via the act of predicting what ought to come next.
Of course, these machines are "surprisingly" good at a lot of things the larger they get, because what the labs have created here is a rough version of humanity's collective knowledge given form and the ability to say hello.
I suspect that the current generation isn't close to the "true frontier" of what these machines could be. They are nowhere close to the sum of all human knowledge and endeavor. They are quite a way there, but they haven't yet achieved true completeness for domains where the data isn't so public.
I think it's the most exciting scientific and technological breakthrough of my lifetime. And I can't wait for us to get close to the true frontier of all domains.
My negative feelings towards AI are about energy use and inequalities, that kind of stuff. It undeniably works well, but whether or not it is better for society or the planet is a lot less clear.
But I want to be reading about that from the comfort of a home, with a full belly.
However, I was looking at the proofs and reason explanation and openAI should be more explicit in how the work has flown. I find the models have jumped hoops in some places of the proofs, that can be hard to track. In fact, when a paper is published you usually get a review and if no reviewer understands they ask you to further explain the thought process. It will be fun to see if this happens here.
Which is very suggestive - if after everything they are not fully loaded then the next gazillion data centres being built look unlikely to be needed.
If anything it proves more data centres are needed. That's literally the only reasonable conclusion from this news.
Are you arguing there is not an AI bubble, and that all the DC buildout is fine, going to be profitable etc?
I am not looking for a online slanging match - just looking for a different point of view
I’m not participating in the slinging match but it’s very very weird that you think it’s some established thing that these companies won’t make profit. A lot of hubris must go in this kind of thought. Like.. do you all think everyone’s playing musical chairs?
Apparently you don't.
"These are real problems mathematicians and computer scientists have been working on and were unable to make progress on."
Who says no one was making progress? Who says openai has made progress? How would anyone not working on these specific problems, witho the time to dig into openai's claims, be able to tell? Why should this not be lumped in with all the other ai hype being pushed?
"The mathematicians I know are saying that the latest crop of models is changing the way people do research math, I think that's a pretty big deal."
Who? And doing what?
We have been hearing the "this generation of models is the one" type talk for years and the only concrete "big deals" are what? A tool for college students to write papers? A replacement for, now enshitified, google search? The fact that now you can fake tons of stuff to support a position or claim tons of stuff that goes against your position is fake?
> Who says no one was making progress?
Let's look at the Jacobian conjecture, since that was the open math problem I was most familiar with prior to its solution. Yitang Zhang, one of the worlds most renown mathematicians (famous for his lower bound on the twin prime conjecture) spent 8 years working on this problem with his advisor (who himself is a renown mathematician) and turned up completely empty handed. His advisor described it as a "waste [of] 7 years of his own life and my time" [1]. Of course, these two were not the only ones working on this problem for the almost 100 years its been open, but they should have sufficient credentials to show that they were not fools or amateurs.
And in a single afternoon an LLM disproved the conjecture. How is that not an extraordinary feat of technology?
> Who? And doing what?
A close friend is studying differential geometry in a PhD program. Sadly I doubt anything I say on his work will convince you, so I will instead offer two anecdotes:
Terrence Tao (widely considered the worlds greatest living mathematician) has said AI is precipitating "a crisis in the foundations of mathematical values and practices" [2].
Timothy Growers (fields medalist & one of the leading researchers in combinatorics) has said that the latest models are now at the point where they are "producing a piece of PhD-level research in an hour or so, with no serious mathematical input from me" [3].
You can find many more fields medalists and mathematics researchers with the same impression. If you look in this thread you can see bluesky/twitter threads from those who were actively researching some of these problems who are in shock at the solutions.
[1] https://www.math.purdue.edu/~ttm/ZhangYt.pdf [2] https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.p... [3] https://gowers.wordpress.com/2026/05/08/a-recent-experience-...
Thank you for the thoughts and references.
There was definitely at least some progress on the problem. I get the general sense that there were potential counterexamples that were close but not quite enough, and that its possible (or even likely) that Claude built on those in order to construct its solution. I also get the sense that when Zhang was working on the problem it was believed that it would be proved true, but since then there were bounds found on the problem that pointed researchers to believe it was false. I am not a research mathematician in this field though, so I could definitely be wrong.
Also, in fairness to Zhang, I believe the dissertation he ended up writing was focused on the 2D case in particular, which is still unsolved (the counter example is only for 3D and above). I cannot imagine that anyone looking at the 2D problem was not also looking at the general case as well though.
[1] https://x.com/henryquantum/status/2083623695436623915?s=20
And yet this is the exact same company that has screwed up their android app so bad that the latex N^3 rendering problem makes it so having it explain it to me crashes the app.
Truly jagged beyond belief.
Mathematicians will tear it to pieces if any of it is fake!
These companies desperately want a return to serfdom. If they didn't come off as so anti-human the public backlash wouldn't be so great.
This was not a problem that was for sale
Look at this two threads.
I'm sure they must do some of this type of work, right?
What I had hoped for was something more ambitious: using AI as an arbiter in economic, political, and social debates, one capable of weighing evidence, exposing trade-offs, and helping us make decisions that produce better outcomes over the medium and long term, even when those decisions conflict with powerful private interests.
I suspect, however, that this is not a particularly urgent goal for the people funding and directing these systems, many of whom live far removed from scarcity and its consequences.
I agree, some of the problems are more difficult. I don't think that's the case for all of them. And, besides, these companies could be demonstrating how to approach problems and where their users could spend tokens to help with these problems.
Should not these companies try to work on these problems _because_ they are difficult?
We already know how to solve all of these issues. What we lack is collective political will.
And yeah, the lack of collective political will sucks. It would be naïve, however, to think that there is no value in ensuring longevity in our current and future infrastructure. And improving it to sustain the population giving these companies their value is an obvious win.
This is division of labor, and it's a good thing. I'm sure OpenAI employees, who are very well paid, donate some money from their salaries to others working on the areas you're citing: probably more than you think, I say that from having attended some EA parties back in the day.
But that isn't my point: my point is that a company which makes brushless motors should put most of its time and money into solving the "make and sell brushless motors" problem, and if they or their investors feel like they need to do more for the world, give money to the people who have the time and ability to do things about that. There are a lot of quality-of-life improvements which need brushless motors.
Next question is how useful their product (OpenAI, I mean) is to more focused do-good-in-the-world professionals. I'm sure that varies quite a bit. For getting the homeless off the street? I conjecture, not very useful. For 'complete the transition off fossil fuels'? Extremely useful, no one in that field knows how to do their job without AI in summer 2026. I'm certain of this.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
Sam Altman has said "If superintelligence can't discover novel physics, I don't think it's a superintelligence." Is that the test? How far away are we from AI discovering novel physics? It seems within reach.
Just maths isn't really general enough for the G in AGI.
AI development is likely to be more compute bottlenecked than solving math problems since validation of any algorithmic improvement would likely require significant compute. But you could imagine that at this point it could be economical for a frontier lab to task 10,000 agents to work non-stop on finding novel algorithmic improvements then validating the top 50 out of 1,000 candidates on a GPT-2 sized network.
I would suggest RSI is now very close. The singularity could be less than 6 months away. I'm not saying I'd put a high probability on that, but I'd give it at least 20%, and I'd double that if looking 12 months out.
I know I'm just a crazy man shouting at the clouds, but please take to the consequences of this seriously. I understand that for whatever reason AI risk seems abstract and doesn't seem real, but this should terrify any person thinking logically about where this could all be heading.
We haven't even solved the most basic AI safety problems yet. RSI right now would almost certainly result in an extremely bad outcome for humanity.
I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which would more likely require months/years, but algorithmic progress would likely saturate quickly. I guess it depends on how much you think further AI progress depends on hardware vs. software.
Imagine humankind meets another race, another race shares it's scientific knowledge and humans accept it without experiencing process of discovery. In that case do we really got this knowledge? If we follow machine discoveries like we follow problems in textbook then we acquire knowledge but we don't discover anything. We follow.
There whole lot of philosophical questions that aren't attacked now. Are complex systems sentient because consciousness is emerging behavior? Then should they have rights? Philosophy is part of humanities and science (is/used to be) part of philosophy. Should we accept non-human knowledge in science? Maybe it's altogether different thing from science, yet very similar.
Is there good historical data on some measure of strength across representative populations over time in the modern era? I'm doubtful.
We do know that the introduction of agriculture diminished strength:
"Bone mass was around 20% higher in the foragers - the equivalent to what an average person would lose after three months of weightlessness in space.
After ruling out diet differences and changes in body size as possible causes, researchers have concluded that reductions in physical activity are the root cause of degradation in human bone strength across millennia."
cam.ac.uk/research/news/hunter-gatherer-past-shows-our-fragile-bones-result-from-physical-inactivity-since-invention-of
What can we do to make conversations around these incredibly exciting and important topics more constructive? HN is where I expect to read expert comments on these topics, has this style of conversation moved elsewhere?
None of this is anything out of the ordinary; this kind of thing has always happened. The only real story here is that moderators sleep sometimes.
I for one am really fascinated by AI's advances in science and math and would like to talk about it somewhere without the constant flamewars...
People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps leading to crazier and crazier results. The much more interesting question to me is what will be consumed by the exponential like math seems to be, and what won’t. Writing has been much more stubborn, but I’ve noticed Fable to be quite a big step up there as well. How about politics? Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation?
And then there’s questions like, even if AI can answer increasingly complicated math questions, will we still need mathematicians to translate results to the real world, verify them, or decide where to push the frontier?
For people who have been paying attention to accurate predictions leading to our present state of the world, HN has collectively been reactionary, incorrectly dismissive, and incredibly behind the curve. In public, people are better informed on LessWrong and AI Twitter circles. That's where frontier researchers are. Barely here
Just glanced at it and it looks pretty good!
Yes. And there are many of them. I wonder what would help them come to terms with it. Seriously, people are going to be grieving over this. Loss of identity, loss of social standing, ideas of entire future lives that will now never happen. The greatest crime people may hold AI guilty of is taking away their dreams.
I think recognizing and accounting for your own personal biases is one of the requirements of the being an intellectually honest and rigorous online discourse participant.
Things could be genuinely impressive and fascinating even when directly challenge your ego and material well being.
Moreover, mister elite, you don't know why this press release was flagged.
I'm not sure why we should privilege your bitter speculation over more mundane possibilities.
Moreover, if it wasn't flagged at all, like you suggest, then the grandparent was inventing things to be bitterly resentful about... Which is not a behavior any forum should indulge.
It was always mainly a website for employees of an elite.
Is that possible or is everyone else too common to have those?
eg. this: [flagged] A migrant surge tests Spain's open policies (economist.com) - https://news.ycombinator.com/item?id=49131860
is clearly marked as flagged.
Unlike the current submission: Ten advances in mathematics and theoretical computer science (openai.com) which isn't [flagged].
> if you compare its rank to that of other stories with a similar age and number of points.
Ranking is complicated enough here even before weighting, speed of initial upvotes can play against ranking, number of comments and the shape of the comment tree also affect ranking. And yes, various subjects and submission sources do get weightings that impact ranking.
What's funny, to myself at least, is that any attention at all is paid to "HN front page ranking" - I've been on again off again active here since 2008 .. and can't recall ever really looking at a default HN "front page" ever.
( There's /newest /newcomments /active etc to browse and sites such as https://hckrnews.com/ )
hah, sorry, we are plebs out here.
https://developers.openai.com/cookbook/examples/vector_datab...
How are the sales going?
It's more like they've already automated the parts of the jobs that the humans most closely thought of as the "their job"
My guess from following this stuff quite closely is that these companies are still a couple years away from fully autonomous research staff.
[1]. https://www.anthropic.com/institute/recursive-self-improveme...
I think we've now hit a point where 99.9% of the population gloss over these types of AI advancements because of human competence being insufficient
No human could have published this because it requires paradigm shifts (e. g. Section 5) in multiple mathematical domains. Mastering one of them to this degree is rare, mastering 3+ pretty much non existent for humans.
For some reason comments got moved to this one.
If they're publishing the solutions to these 10 problems, then this is, essentially, the announcement of 10 research papers.
Yes, it's partly for reputation (as are many research papers), but that doesn't mean it's of no value. The best way to advertise is to show that you're providing value.
Stop coping and deluding yourself mate.
To begin with, whether AI is the one doing the discovering or not makes no difference. Any "pure math" person would aim to understand regardless - and would be quite glad that they have a longer paved path.
Any mathematician in academic or industry is more than likely not a "pure math" person (tainted by capitalism).
>Any mathematician in academic or industry is more than likely not a "pure math" person (tainted by capitalism)
Ignoring that I meant pure as in non applied math, let's just make it clear: you agree that mathematicians who are against capitalism encroaching on this process should be allowed to dislike it without criticism of being pretentious?
Isn't coping a good and useful mechanism?
https://garymarcus.substack.com/p/openais-amazing-but-vastly...
https://garymarcus.substack.com/p/two-critical-updates-re-as...
Not that there isn't something interesting in here, but lets be clear that we don't have enough information to evaluate this properly. And as always with these labs, BS takes a lot more energy to refute than it does to spread.
> Astra, a new model that OpenAI is testing internally, is amazing. No denying that.
To sharpen that, I think he's (obviously) interested in maintaining his own brand as "thought leader" and this necessitates de rigeur defense of particular postures.
Sometimes this is easy because the facts warrant it; other times, a bit of rhetorical license is required to preserve nominal coherence and (at least, for the moment) hold certain lines.
This is one of the latter cases, and it's not subtle.
One of the celebrated properties of many intellectual advances or inventions in whatever domain is precisely that it appears obvious in hindsight. It is quite cynical to leverage consensus distrust of large AI players, warranted but also a popular social construction, to insinuate that these are not "real" advances or "real" hard problems, on the grounds they were in some sense cherry-picked.
Identifying the problems amenable to strategies on the table and intuitions (sic) about where bridges might be, is exactly the discerning work that is the core driver of almost all prior progress, but for celebrated accidents and flashes of insight. Anyone working in any challenging discipline knows that those are celebrated and told around campfires precisely because meaningful durable results arising like that is so uncommon.
These two articles make me think of nothing so much as my own durable reaction to the creeping goalposts of AI critics generally: that they often seem to me not unlike a water color cohort scoffing and jeering at the horse, because it got a D on its tensor calculus exam.
Marcus should be on guard against his own cynicism and take care that his assumptions do not prevent clear sight.
I think that is an oxymoron.
""" 1. By 2029, AI will still be unable to watch a movie and accurately explain the characters, events, conflicts, and motivations.
2. By 2029, AI will still be unable to read a novel and reliably answer questions about its plot, characters, conflicts, and motivations beyond what is stated literally.
3. By 2029, AI will still be unable to work as a competent cook in an unfamiliar kitchen.
4. By 2029, AI will still be unable to reliably create more than 10,000 lines of bug-free code from natural-language instructions or interaction with a nontechnical user, excluding simple assembly of existing libraries.
5. By 2029, AI will still be unable to convert arbitrary mathematical proofs written in natural language into symbolic form suitable for formal verification. """
There's still 3 years to go and he's already wrong on 4 out of 5.
1. still not wrong? Unless it's just feeding the audio or screenplay I don't think you can feed AI a full movie in a single context window yet?
2. Not sure, but can you prove this wrong? Can you feed a full, unseen new book and get that kind of answer?
3. Not wrong.
4. I think he'd probably pull you up on 'bug free' - I don't think that frontier models can reliably write 10k LOC without _any_ bugs typically (not that humans can do this either).
An LLM could theoretically try to earn some money and pay a human to do most of these tasks but it's not the point of the exercise.
Who is going to wade through this?
The non-sofic group one is definitely a big deal - would have been a Fields medal if discovered by a human.
AI has no self-awareness. It's a tool. When you assemble a furniture using a screw driver, the torque force interacts with the molecular forces inside the metal and miraculously it transfers the force to the screw though a clever geometry design, communicating the force to the screw to turn it in a certain way.
Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down.
A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt.
[0]: Not one of the proofs in the linked article, but from OpenAI too.
I think you're over-estimating what a smarter highschooler could write.
A "finite loopless undirected multigraph" could have been explained to me at that age if we'd taken Discrete rather than Mechanics and Pure (and one module of Stats) in my two A-levels* in maths and further maths; but from what I saw of the Discrete module, neither:
Every finite loopless multigraph with no bridge possesses a cycle double cover, without additional assumptions such as cubicity, planarity, connectivity, or higher edge-connectivity.
repeated-edge closed trails masquerading as cycles
* For those not from the UK: A-levels are between secondary school and university, when aged 16-18. Functionally they are university entrance qualifications: https://en.wikipedia.org/wiki/A-level_(United_Kingdom)
Provability is just going the way of computation. John Napier had to manually compute logarithm tables over decades and was recognised for his work; now that same work could be performed by a 10 year old with a calculator in an evening.
> In particular, proofs for special graph classes, constructions of cycle covers with some edges covered other than twice, bounded-length or prescribed-cycle variants, reductions to another unproved conjecture, computational verification through any fixed graph size, and candidate counterexamples without a complete nonexistence certificate are insufficient.
which is infact a very important part of the prompt.
Take a grad student with a perfectly good understanding of what a proof is. Their supervisor gives them a major problem to work on. Almost always, the problem is too hard, the student comes back with partial results, and student and the supervisor iterate from there. Now imagine that they have an unusually cruel and unreasonable advisor who tells them, do not dare to talk to me until you've fully solved the problem. This paragraph is exactly that. It's there exactly because the underlying system is smart enough to know that real mathematicians do not work like that.
To a real mathematician you would not have to list those explicitly, he/she would have understood that implicitly from 'give me a full proof'. That listing makes sense to say only to somebody who pretends to be a mathematician, but has not true understanding of how the math works. The models are getting better and better in this pretension, but prompts like that reveal that it is still just a pretension, not a true understanding.
To name a few:
- https://xcancel.com/DmitryRybin1/status/2079904005652893709
- https://archive.ph/2w4fi (https://chatgpt.com/share/69dd1c83-b164-8385-bf2e-8533e9baba...)
I also have a 3D printer. I also have a ChatGPT subscription, and some OpenSCAD chops. I came up with a part which would go into the top of the down tube to the drainage pump, and mostly-seal the down tube itself, with an opening on the side to vacuum out the moisture. This was purely prooompted, I took some measurements, printed bits of the part, refined the shape, and you know what?
It works! I can stick it down there, turn on the (very loud) wet vac, and go upstairs. On a 1.5Ah battery it sucks for a bit less than ten minutes, which turns out to be plenty of time.
So: who made that?
Don't care. I'm waking up warm at night.
Also: me, obviously. ChatGPT doesn't have a fucking furnace.
When the tool is a 3D printer, or any CNC system really, you bet I attribute a build to it.
I could also attribute the operator; there is no contradiction, it's a free choice, just like saying "I am in Berlin" does not contradict "I am in Germany".
What is your mechanistic model of self awareness that yields this conclusion?
> It's a tool
Does your model suggest that tools can't have self awareness?
(*) Even if we hack around this and just do the usual trick of simply laundering statefulness to a higher level, in this case the context window being fed in, I fail to identify (**) a representation of its own state in these bodies of text that it'd be meticulously maintaining. I further fail to identify how it could be hidden or maintained, considering I control like half of it. The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line").
You'll sometimes catch models mixing up who's who and how many who-s there even are for example.
(**) I did wish for something hidden though, so maybe it's just concealed? The same way people can encode a lot more of their emotional and mental state than normal into text if they read and write a lot of it, I'm aware of research that suggested the same for LLMs, albeit I cannot cite it. Maybe those phrasing signatures are just alien to me and will never pop out. Either way, I'd expect researchers to stumble upon this during interpretability studies, and either they haven't, they have but it wasn't popsci adopted, or they're keeping awfully tight lipped about it. If you know of anything like this, your turn now, would be happy to learn.
I do wonder how reasonable it is to expect e.g. a single maintained identity though. Maybe it isn't?
(*) Another way to hack around this of course is to just precompute some internal "self-awareness states" and hop around between them. Probably the closest to what the models are actually "doing".
> a hidden representation of self that is continually tended to
This sounds like a personality? They act like they have one of those. It may be an illusion, and even if it isn't an illusion it is unlikely to be anything like the source (us), but they act like it.
> I further fail to identify how it could be hidden or maintained, considering I control like half of it.
Indeed you control everything about a local model, and much of the context of even a remote model. But the state of activations and circuits in SotA AI is hidden in similar ways to those of synapses in your head: difficult to decipher even with probes monitoring the signals directly, and often not emitted at the normal output.
> The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line").
While we can be confident that LLMs make up personas etc., it is insufficient to go from "that doesn't necessarily represent the model's internal state" to "therefore it doesn't have one".
> You'll sometimes catch models mixing up who's who and how many who-s there even are for example.
I've, unfortunately, also experienced this with humans. Perhaps they were losing their self-awareness at the time? I do wonder if old-age dementia does that by the end, though the person in question didn't ever get diagnosed with that.
> If you know of anything like this, your turn now, would be happy to learn.
Do you mean like these, or something else?
• https://researchportal.hkust.edu.hk/en/publications/decoding...
Not quite what I meant, but it's also not entirely unrelated I guess? Personality to me is like a natural bias. It does also shift over time, and is also an internal bit of state. I guess in some respects it can also be self-referential, like personal convictions.
> Perhaps they were losing their self-awareness at the time?
I do think it is entirely possible for people's self-awareness to shift, yes. Or more precisely, I do model things that way.
> Do you mean like these, or something else?
They're adjacent, but I more meant something like these:
https://arxiv.org/abs/2410.03768
https://arxiv.org/abs/2310.18512
https://arxiv.org/abs/2605.26537
So basically, steganography. The difference is that these papers investigate from the perspective of separate LLM instances covertly exchanging information between each other. This is in contrast with the scenario I'm laying out, where an LLM's past state is exchanging information with its future state, continuously representing and modulating a concealed internal state of some sort. And then that state just so happening to be some sort of self-referential meta state.
And the best inkling I have towards this is basically: https://www.youtube.com/shorts/WP5_XJY_P0Q
But then I don't think there's enough covert channel bandwidth in the agent replies for anything interesting like this.
I don't see why an LLM could not have a sense of identity or personality while it's evaluating a specific prompt, or even change self awareness while evaluating a prompt since many outputs model a back and forth conversation. My point is that without a mechanistic model of what "self awareness" means, we have no way of truly evaluating such questions, we're just hand waving vague intuitions about what it could mean.
This is kind of also the reason e.g. the HN site guidelines are worded the way they are. Regrettably, forums naturally yield themselves to tit for tat type exchanges, but there's really no reason one could not bounce such vague intuitions off of another. I do not have to be right or wrong, and you don't either. Admittedly difficult when its some intensely contentious topic.
If a mechanistic model existed, there would also be no reason to talk about this in the first place. There'd be nothing to discuss, you'd be simply told how a given model characterizes from this perspective on the model cards.
The reason I insist on mechanistic models is because the original post was making a definitive knowledge claim, and in my experience, the knowledge claim is unwarranted.
A language model (or an image model or whatever) cannot even be sentient, and I think sentience is a prerequisite for awareness.
Even if we express a lot of our subjective experience with words, the language is just a symbolic representation of those experiences. The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.
You can't have an understanding of what hunger or physical pain feel like if you have no need for food or a sensory capacity for feeling pain. You can't understand what loneliness or pride at an achievement mean if you don't have a neural network wired to value social connection or status. We value connection because we're social animals that have needed each other for survival.
Even the more abstract of our subjective experiences are in some way rooted in our physical evolution.
I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.
AI awareness might actually be more believable if that awareness manifested itself in an entirely different way than in humans. But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience, I think we're seeing something that isn't actually there.
There is no objective evidence of qualia. All evidence of qualia are vocal or other expressions of belief in qualia. Perceptions clearly exist and are observable, subjective experience and qualia, not so much.
> I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.
If your objection is to models based on "symbolic level of language" which you think lack semantic understanding of, say, trees, you should ask yourself how our brain, based on physics which also lacks any semantic category for trees, can somehow develop a semantic understanding of trees. All of these appeals to differences with the brain never seem to acknowledge that fundamentally, the brain has the same explanatory gap with physics.
> But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience
This assumes a lot. It seems very possible to me that intelligence inherently develops a map of natural categories (natural kinds), and language naturally develops around such categorical understanding. Semantics are then fundamentally the network of associations between categories, eg. there is no fundamental difference between symbols and semantics, and the latter cam be inferred from the former, and that's exactly what LLMs do, and why the semantic maps between different languages are so similar and how they can translate between languages.
You didn't address any of what I wrote, let alone provide any counterarguments. Which part of what I wrote do you think was wrong?
For example, the Turing machines and the lambda calculus don't look anything alike, but they are fundamentally interconvertible, and so in a real sense they are fundamentally equivalent. Without a model, all of your arguments are completely unconvincing for exactly the same reasons, eg. that there may exist many paths to fundamentally equivalent ends.
Hunger as a concept doesn't mean anything without the physical need. Politeness or bluntness, even in writing, don't mean anything without social dynamics. And we have social dynamics (and neural structures that directly process social cues and associated feelings) because we've evolved into social animals for whose survival that was important.
I see no reason to believe that a model trained only with symbolic representations, with no connection to the physical world phenomena that those symbols represent, could contain the subjective experience itself.
Neural network models may be able to derive novel (or at least novel-looking) output rather than just an obvious rehash of their input, but I don't think any set of bytes can fundamentally contain information that was never entered into it. (Even if e.g. a model produces previously unknown mathematical results, those results can in principle be derived from the information that they were trained with.)
I'm not saying that artificial neural networks couldn't, in principle, be aware. ANNs and biological neural nets may be equivalent in the sense that any information and processing structures represented by a biological one could in principle be represented by an artificial one. If that's the case, and awareness is purely a product of our neural systems as materialism would imply, it should be possible for an ANN to be aware, too.
But when the model has been trained with only language, and IMO the subjective experience can't be derived from the symbolic representation alone, I can't see how the model could include the actual subjective human experience.
An AI model could of course have an awareness and subjective experiences that are totally different than our human experience. But then the fact that it happens to produce output resembling what humans find meaningful shouldn't be considered indicative of such awareness.
This is of course more of a philosophical argument than a technical one, and I'm happy to hear counterarguments, but not on the level of off-hand dismissal.
People who say LLMs have subjective experience aren't saying they have human-type subjective experience. Nobody who sees an LLM express hunger when role playing as a hungry person thinks that the LLM is actually hungry.
I too can role play as a hungry person despite not being hungry, so there is no reason in either case to conclude that the words produced reflect genuine internal subjective states. The point is that such internal states may still exist.
Without a searchable index of training data, it is hard to put faith into these claims.
eg. "we spent another 2k on searching for pre-existing proof but found only the weaker result xyz by abc in 1972" would be in the spirit of academics quoting prior work.
The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation? How about experiment driven domains like biology?
What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.)
Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them.
Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.)
My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.
Another degree of capability, yes. But we have been trending here for a long time.
There's a reason attempted murder is a crime even if it was unsuccessful
We should not be gung-ho to give the government more power to regulate speech.
The point of view that says 'there is a reason we punish attempts' has difficulty explaining why we punish 'success' _even more_. There is a sort of bad concience about it. The paradoxes are discussed in a characteristically brilliant and twisted work 'The Punishment that Leaves Something to Chance' by David Lewis, one of the greatest philosophers of the 20th c. It nominally defends the law of attempt but can as well be read as a catastophically destructive parody of it. https://andrewmbailey.com/dkl/Punishment_Chance.pdf
You're not allowed to bribe public officials. You're not going to escape the law by arguing that the bribe didn't get you the result you wanted.
Ads are not really the same. They can’t be as tightly targeted to what resonates with someone. Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them.
That’s where this is headed and people are not ready. I don’t think we could prepare them, anyway.
> Ads are not really the same. They can’t be as tightly targeted to what resonates with someone.
The EU referendum in the UK proved your point false.
> Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them.
When people talk about “ad tech”, they’re not talking about TV ;)
And yes, people can and do engage with them. That’s how ads on social media works.
During the EU referendum, people were even resharing ads on Facebook without even realising they were ads.
insulin is not straightforward, the way the insulin molecule interacts with its receptor is nuts. on the other hand its post translational modifications are simple and dont have anything particularly surprising (no glycoslation, disulfide bonds where you would expect, nothing special kex2 cuts, arent really defective in disease states even)
Decentralized manipulation, by contrast, just runs amok creating echochambers and polarization.
Side note: it's manipulation either ways because you chose what to talk about, with a goal in mind.
when it comes to the important question, then both "sides" are the same team.
or if you want it with a pinch of humor:
when a boot is on your face, it makes precious little difference whether it's the left or the right boot.
(i lived the first 10 years of my life in communism)
The left in the US would be center-right in Europe, who are certainly not communist (they have separate parties that are communists!)
Even the socialist strain of the US has nothing to do with socialism scaremongering about Venezuela, etc.
The "left" as in the pervasive group that crawled out of Tumblr, took hold of Twitter back in the day, and has a stronghold on Reddit now? Those do care about what you can say, think, watch and read, and the more they can control, the better. The US right can only dream to have half as much control as the left has had in the last three decades.
What's "scaremongering" to you is "life" to GP. Besides missing their point, you're also saying "you held socialism wrong, we can make it work, if only if it weren't for this pesky ... reality"... Sorry, you missed their comment, and I can't take yours seriously :)
I think that statement may itself highlight how prevalent manipulation is.
I fully anticipate all groups to continue maximal manipulation they can. One thing with LLMs is that it'll be a far less unified view, so a "divide and conquer" strategy is what I anticipate.
Look at the most contentious issues in the US: abortion, climate change, taxing the wealthy, gun control, Affordable Healthcare Act.
The Democratic Party platform is aligned with national polling on every one. Every one of those issues has >60% support with voters and the Republican Party has blocked them all.
They play the game to win. And they do.
You left out immigration, crime, and “moral values”: https://www.pewresearch.org/politics/2024/05/23/top-problems...
The same thing for "taxing the wealthy." 59% of Americans think their own taxes are too high: https://news.gallup.com/poll/707951/americans-tax-views-rema.... And the difference isn't as big between parties as you think--49% of Democrats think their taxes are too high. So Democrats are in a position where they have to advocate for raising taxes on "the wealthy," without scaring any of their own voters into thinking that includes them.
Even a blind squirrel could find these nuts. Apart from that, republicans are ridiculously inept. For example, 2024 was the first time they spent real money trying to go after immigrant voters and minorities, and they made huge gains. But the on-the-ground operation disappeared after the election. Meanwhile, democrats are in these minority neighborhoods 365 days a year pushing their message.
They're not. They're really not. They're incredibly capable and are currently executing long term plans successfully, one after another. This goes all the way back to Regan and the disenfranchisement of education. They know exactly what they're doing to erode democracy.
Calling them inept is dangerously stupid at best, and at worst is just right wing propaganda to try and lull people into a false comfort and not act.
> Democratic Party platform is aligned with national polling on every one.
It depends on the pollster and where you're polling. I guarantee you rural Tennessee will not agree with downtown Washington DC on any of these issues. In contrast, rural California will likely agree with rural Tennessee. It's not as cut and dry as a homogeneous national poll of 2500 people. Every state, city, county are different. That's why there are federal, state, city, and county governments.
For instance, Abortions are legal nationally. States can individually decide how, or if, they wish to restrict it. This is as the constitution intends under the 10th amendment:
> powers not delegated to the federal government nor prohibited to the states are reserved to the states or the people.
This allows for democracy to take place at the local level, rather than having particular regions thousands of miles away from each other ultimately oppress the other.
To the point on LLMs, I think it's abundantly clear they will be used to propagandize and similar to social media will lock people in a bubble without alternative opinions. It'll be the worst of both worlds, the question is who's the puppet master. At some point soon, I imagine it'll be the AI.
Which amendment protects someone's bodily autonomy or medical treatment from interference by the states?
If the Federal Congress passed a law providing that abortions are legal on demand up to the 22 week of gestation, after which it would require the corroboration of two doctors, would that stand, or would it be struck down by SCOTUS?
Same for legalization of marijuana, and many other "social" issues.
The pretense of Dobbs is that laws against abortion are like laws prohibiting medical marijuana or setting speed limits. This contradicts the purpose of laws against abortion and the actual language of most of them. It was obviously a desperate decision, irrespective of the truth about abortion
If you've got 5 people in a car and you play ABBA in every road trip, then one day you suggest to change, the problem is not in being okay with "something else", but on agreeing what that other thing should be, set against the already known thing.
That's why there's so much infighting in the left.
They have no profit requirement or even revenue neutral requirement. So they can just operate poorly at a loss and still wreck other businesses because people will put up with breadlines to get bread for ultra cheap.
The general thing to watch out for with all of these “surely it can’t be evil to do nice thing X” is suicidal empathy. It can seem correct to your gut on the surface while it’s extremely destructive in the long term despite participants wanting to destroy something as an explicit goal.
FTFY
Yes, ai has strong narcissistic traits. And so do the people that own them. And pay for them.
In response, people in general will become fat more capable of recognising the manipulation. And will become more paranoid.
I wonder if AI will end up being similar. Certain theorems get proven by AI but others do not. We haven't reached the limits of this yet and I haven't found a good argument for where those limits will be (I do doubt that there are no limits).
> https://www.astralcodexten.com/p/the-sigmoids-wont-save-you
The conclusion of this article seems to be "you should give ai the benefit of the doubt against all reason". Barf
That aside, I’d question whether applying the Lindy effect in particular to something that’s not really a life expectancy but more a growth rate is credible… or perhaps a bit circular since it “assumes away” the ceiling.
The simpler assumption is that over enough time, the S functions stack together for long enough that working backwards from exponential is a better predictor of reality.
These stacked S curves have continually been true with most technology.
No future for research mathematicians othet than as tastemakers / agenda setters?
Nothing in that article says it's not a sigmoid.
We have though. The velocity remains high but the acceleration is decreasing.
The improvement between gpt 2-4 where staggering. to 5, 5.6? Much less so.
The results improve of course, but the difference is no longer mind blowing.
Most of the improvement now has come from agentic harnessing, which is unrelated to the acceleration of the model but the tooling around the use of the model.
So we are seeing a deceleration and the speed is still high due to high prev acceleration but its not growing at the same rate as before and the edges are starting to show themselves
The diminishing returns are colossal. GPT 4 is more than 3 years old now, and while Sol is definitely better, it’s better at a tremendous cost. And 3 years of development. And it still fails in nearly all the same places as 4.0.
I’m not some “zero AI” person, I don’t think this industry will collapse into nothing and we will go back to a no LLM world.
But to look at this industry, which basically began in November 2022, almost 4 years and a trillion+ dollars later and say “oh yeah, definitely riding that exponential growth still” is some of the wildest shit I’ve ever seen, and it’s so common.
A lot of this sort these sorts of posts are just "appeals to geometry" (aka "cope"). This is coming, it's coming hard. Now you need to decide what you want to do with your life in a world where your smarts aren't as special as they used to be.
This is hard (believe me, I know). But what one ought do is not eschew progress and cling to the delusion that things don't change, what one ought to do is try to see how they can leverage these tools for greater and greater accomplishments.
In other words, big labs are much more interested in making "AGI" than in making a good writer, especially as what qualifies as "good writing" is rather subjective. E.g. before AI use of metaphors and rhetorical devices were generally a sign of a good writing. Of course, not if you keep spamming the same rhetorical device - but a stateless AI can't know which one it is over-using.
Eg.a 20usd/m plan usually don't cut it for professional work.
Overall I'm excited for this acceleration in discovery, even if it's causing disruption to existing research workflows. I wrote a bit about it recently [2], after seeing Levent Alpöge's counterexample to the Jacobean conjecture.
[1] https://www.cnbc.com/2026/06/30/anthropic-launches-ai-drug-d...
[2] https://scholarlyfutures.substack.com/p/frontier-models-tran...
Both are the same thing. Two sides of the same coin.
However, I also expect this squeeze will come at an increasingly expensive price — not just because of inefficient token usage, but because of fundamental limitations of LLMs as a model.
LLMs are letting us brute force our way through a lot of reasoning, but it’s hard to believe that such a generic model of intelligence will take us to the next frontier. We’ll need some fundamentally new approaches at some point. Maybe those will make achieving the exponential more efficient or maybe they’ll unlock even higher degrees of possibility. Who knows?
I think a nice example is using them for arithmetic. It's a specialized deterministic process, so it's extremely wasteful to do it that way.
But they're good at finding solutions to things we don't know how to specialize yet.
So, to use metaphor, maybe the transformer-based models are like the FPGA, and then when we figure out the patterns in that system — all the different kinds of specialized reasoning — we can extract it into an ASIC?
We know language has to be learnable by every human, so it needs to be really independent of any specific brain development particularities. If it was not accessible to babies there would be no more language next generation.
Many many things are only useful when expressed in the physical world, and that introduces lag.
In general, you can think of the process as generating massive rollouts in generation N, and then compiling in the verifier/human feedback("gradient") signal into generation N+1. The time taken to make the rollout in generation N, and separately the time taken to get the same rollout in generation N+1, each grows constant in some tasks, linear in more, and exponential in some.
In the end, this becomes bottlenecked by time. Today, we can make statements like "I generated all these successful trajectories with 2 weeks of compute, in the next model it will be able to do it in 7 hours of compute", but very soon you'll find yourself making statements like "I generated.... with 8 months of compute, in the next model it can do it in 6 months", which isn't really enticing the same way you can _technically_ brute force passwords but it just needs prohibitive amounts of time and money. That is the "plateau". Note that, this point is quite far away. For example, at any point if we agree it plateaus, today's known hardware techniques such as fixed function accelerators give you a 10-100x timeline reduction immediately allowing for a few more cycles of improvement. This is not to mention future innovations, but of course none of that is helping with the benchmarks where the time needed is growing superlinearly.
In many math and coding benchmarks, we are still in the constant phase. These are the massive improvements we see every few months. I'm not making any prediction of what will plateau and what will not as it's not possible to make an informed prediction about these things IMO. But the observed fact is that some have already plateaud as in, they don't improve with reasonable inference time (likely superlinear growth).
> will we need mathematicians to translate
Let's take a sudoku analogy. The model is initially just doing the random value algorithm, but lets say you the human are watching it. You make one of the usual reductions and interject "hey you can stop trying 8 here because of ....". Over enough examples, you get to a point where the model is _forced_ to learn the logical pattern. Next generation, it will skip that number. After this, you can peak the distribution using simple 1/0 RL. Doing _pure_ 1/0 RL works decent, but its not frontier as its a very sparse signal.
For that lift, human (or even a better LLM, but if you're trying to improve a frontier LLM, there is by definition no better LLM) feedback becomes necessary. This is _why_ it is crucial that these models interface in natural language and is also why the labs are hiring AI tutors by the hundreds. The "better LLM" case is what Kimi etc are doing by "distilling"(bad term for this) claude.
> But the long term is completely bewildering if you believe any of these trends can continue at a similar pace for the next few years.
For math and coding, for now we are in the phase where the times are just ... constant, so there's little reason to think it will stop soon. We still need humans to expand the frontier. It just becomes a matter of if its worth the cost of compute for running this generalized The Algorithm or not.
Given how well chess players internalized _many_ (not all) of alphazero's emergent chess knowledge, I am confident we wont have too much trouble figuring out any new math LLMs come up with, which will let us keep expanding the frontier by giving the LLM the next "lift". Only when we reach the stage where the time growth become exponential will this stop, IMO.
isn't it clearly split between verifiable not verifiable ? what is interesting about that question.
Programming has verifiable and non-verifiable aspects. Competitive programming, passing tests, and performance can all be verified. But translating English requirements into actual software, software architecture, taste, or UI design cannot. And yet over the last couple years we’ve seen huge lifts in all of these areas, not just the verifiable ones.
Verifiable areas I think are clearly seeing the most improvement, or are the quickest to see improvement. But we are seeing lots of progress in non-verifiable areas as well.
How much of the non-verifiable progress is a function of labs purchasing expert data vs. the models improving with compute is maybe another interesting question, but fundamentally I don’t see spend on expert data as something that can’t grow if AI revenues keep growing as well. And as models get better taste they can also help filter and generate new synthetic data for their next versions to train on. The limits of this approach are not so clear.
most gains are still coming from data. isnt that supposed to 'run out' though?
You could view this as just continually patching a leaky ship. But it seems to work.
I thought this is mostly RL data. In my previous comment i was referring to pertaining data.
Imagine a hypothetical oracle, call her MyladyMath, imagine you can turn to MyladyMath, submit a correctly formed dossier of axioms and definitions, theorems with proofs, and then a newly putatively proven theorem T. MyladyMath will complain if your dossier is malformed, and point out where and why. If the dossier is not malformed it will eventually read in the claimed theorem T, evaluate its proof and then either point out a which step is erroneous and why, or ultimately accept the proof.
Instead of a large corpus of human authored text, this map from dossier/theorem -> accept / reject is a huge implicit array of bits, something fundamental, and this weird gigantic array of bits that effectively describe all accept/reject responses MyladyMath would return exactly. We would never run out of "corpus" when it comes to math, if humans had access to such an oracle.
And we do have access to this oracle, and possess compact algorithms that describe the accept / reject bits. One of them is called MetaMath, a minimalistic verifier, which keeps the concepts of prover and verifier separated, this choice results in concrete proof objects (a sequence of step label references).
The machines are going to comb through all possible paths of the next N steps, for progressively larger N, efficiently compress those results in the weights of an LLM and then use the gained experience as the intuition for guided "not-so-brute"-force proof search, using the prior iterations intuitions to grade the surprisal of the N+1't iteration of results, etc.
The money will not stop flowing in that direction: all power blocs, nation states, militaries, banks, ecommerce, ... depend on cryptography. And the machines will soon do more rigorous proof search grounded in more balanced and objective observations. There is no responsible disclosure mechanism for flawed hardness assumptions in cryptography. It's going to get rocky, and the common man will wonder why the gods have gone crazy, wonder why they don't just pull the plug out of the machines, but nobody will in a staring contest to see who dares keep the plug in the longest (and dominate global cybersecurity).
It's the end of the age of artisanal mathematics, it will now become industrialized mathematics.
This is _much better_ data than 1/0 verification, it is as good as a gradient.
Automatically verifiable tasks improve faster since well, its automated.
I think you have completely misunderstood what OpenAI have accomplished here. Almost certainly no one cares about the specific concrete results achieved; they only care about (a) how difficult it would be for an intelligent human to achieve the same feat (ETA: the feat is the proof), which can be estimated by the amount of time the problem has remained open/a public conjecture, and (b) how general this artificial "intelligence" appears to be, which can be estimated by the diversity of topics where it was able to prove a difficult result.
It's as if I showed you a dog that I had taught to speak German fluently, and you remarked: "What point is a dog that can speak a language that less than 2% of the world speaks? Nothing to see here."
being lucky enough to receive the opportunity of hiding in some academic closet, poking your hand out begging for scraps, was a different, perfectly alternative path humans decided to take instead.
It's a bit late to start standing up for your rights when the robot overlords arrive.
And you’re thinking this is an accurate comparison?
One example that always bugs me is when people point to "exponential" or "sigmoidal" progress on benchmarks. Benchmarks are artificial constructions (saturation at 100% by definition) and benchmark scores should not be mapped to these words when talking about overall progress.
Example - progress on ARC-AGI-3 at the moment is exponential, steeper than 2^t and e^t. Does that mean AI is progressing "exponentially" in the colloquial sense? No, it doesn't support or refute that colloquialism.
Likewise with MMLU saturation. We can't go above 100% by construction. Therefore we have a "sigmoid". Gah.
The colloquialism is not helpful to begin with.
The reason is that the original scaling axes (parameters, training tokens, test-time compute) have saturated already, but RLVR (reinforcement learning from verifiable rewards) is still scaling well. And math has this nice property where you can synthetically generate arbitrary volumes of rewards to train the model, because math is self-contained and completely objective. Open-ended reasoning and analysis don't have that convenient property, and that is why progress is much slower outside of math and coding.
Math ability also helps with other things like making models more efficient.
what did you notice ?
I literally roll my eyes and cringe quite often at its output pretty much daily.
I don't like to overload my sessions with skills but I've been using a "write-normal" skill I made just to have it rewrite outputs that particularly piss me off.
https://gist.github.com/alasano/1c734fa055231a5defcfd213217e...
I'm sure there's a million of these skills out there, but this one is tailored to the stuff that makes me mad in particular.
That said, Fable is still not a great writer, largely driven by it not knowing what it should exclude, and it still having the usual LLM-isms. But it’s better.
5.6 Sol is great but there's a depth to the understanding that Fable exhibits that's unique to it currently.
Can I truly quantify this? I don't think so. Just that I spend a ton of time with various models and a certain point it's just a personal impression or a gut feeling.
In the days after Fable first came out I increased the amount of parallel planning of tasks that I was doing by 2-3x because it felt like I didn't need to be paranoid due to that handling of nuance.
Is there even the tiniest reason to suspect that the people steering this progress will use it for the democratic good of all?
Wealth in terms of capital doesn't represent material goods, it represents the system's confidence in your ability to direct capital efficiently. But eventually efficient capital bottoms out at consumable goods. Someone like Musk with a lot of capital under his control is contributing to the end goal of unlimited abundance.
In a free and fair market, his capital would be regarded as a deeply inefficient distortion.
Yes.
The results OpenAI demonstrated are impressive, but it also looks like they threw a lot of compute at it just to get results. How many tokens did they waste on problems they couldn't solve? Applying inference infrastructure on a large number of math problems at scale we haven't seen before to me doesn't demonstrate an exponential curve in model abilities.
I was recently listening to BBC Radio 4's episode on the Poincare Conjecture[1] and the guests on the program were discussing how the problem that looked deceptively simple eluded the great mathematicians of the time (including Poincare himself) for nearly a century and how Grigori Perelman cleverly came up with the proof. It took other mathematicians working in groups years after Perelman's publication to understand and validate his proof. The mathematicians on the program were speaking of highly of his proofs and admiring the originality of his work. This made me think of one neat experiment where if we cut-off a frontier model's training data 2002 or anytime before Perelman posted his proofs on arXiv and check if it can come up with the solution by itself. That would surely be a great signal to see if these LLMs aren't just solving interesting puzzles and that they can came up with something truly novel.
P.S I highly recommend Misha Green's "Perfect Rigor" for anyone interested in the history of the problem and the genius behind the proofs of the conjecture - Perleman. I found it an entertaining read and could digest its description of the problem as a layperson (with undergrad level math).
[1] https://www.bbc.co.uk/programmes/p0038x8l