Amateur armed with ChatGPT solves an Erdős problem
66 points by pr337h4m 10 hours ago | 17 comments
https://www.erdosproblems.com/1196

adamgordonbell 35 minutes ago
Here is the chat:

    don't search the internet. This is a test to see how well you can craft non-trivial, novel and creative proofs given a "number theory and primitive sets" math problem. Provide a full unconditional proof or disproof of the problem.

    {{problem}}

    REMEMBER - this unconditional argument may require non-trivial, creative and novel elements.
Then "Thought for 80m 17s"

https://chatgpt.com/share/69dd1c83-b164-8385-bf2e-8533e9baba...

reply
ipaddr 21 minutes ago
Tried the same prompt and ended up no where close on the free plan.
reply
userbinator 6 minutes ago
The LLM took an entirely different route, using a formula that was well known in related parts of math, but which no one had thought to apply to this type of question.

Of course LLMs are still absolutely useless at actual maths computation, but I think this is one area where AI can excel --- the ability to combine many sources of knowledge and synthesise, may sometimes yield very useful results.

Also reminds me of the old saying, "a broken clock is right twice a day."

reply
Eufrat 17 minutes ago
Humans and very often the machines we create solve problems additively. Meaning we build on top of existing foundations and we can get stuck in a way of thinking as a result of this because people are loathe to reinvent the wheel. So, I don’t think it’s surprising to take a naïve LLM and find out that because of the way it’s trained that it came up with something that many experts in the field didn’t try.

I think LLMs can help in limited cases like this by just coming up with a different way of approaching a problem. It doesn’t have to be right, it just needs to give someone an alternative and maybe that will shake things up to get a solution.

That said, I have no idea what the practical value of this Erdős problem is. If you asked me if this demonstrates that LLMs are not junk. My general impression is that is like asking me in 1928 if we should spent millions of dollars of research money on number theory. The answer is no and get out of my office.

reply
resident423 15 minutes ago
I wonder if the rationalizations people come up with for why this isn't real intelligence will be as creative as ChatGPTs solution.
reply
walrus01 3 minutes ago
For one, everything its 'intelligence' knows about solving the problem is contained within the finite context window memory buffer size for the particular model and session. Unless the memory contents of the context window are being saved to storage and reloaded later, unlike a human, it won't "remember" that it solved the problem and save its work somewhere to be easily referenced later.
reply
homo__sapiens 3 minutes ago
Big if true.
reply
tomlockwood 25 minutes ago
My big question with all these announcements is: How many other people were using the AI on problems like this, and, failing? Given the excitement around AI at the moment I think the answer is: a lot.

Then my second question is how much VC money did all those tokens cost.

reply
gdhkgdhkvff 15 minutes ago
Why do you care about either of those questions?
reply
Eufrat 12 minutes ago
I think we should at least ask the latter, if it turned out it cost $100,000 to generate this solution, I would question the value of it. Erdős problems are usually pure math curiosities AFAIK. They often have no meaningful practical applications.
reply
jasonfarnon 3 minutes ago
Also, it's one thing if the AI age means we all have to adopt to using AI as a tool, another thing entirely if it means the only people who can do useful researchers are the ones with huge budgets.
reply
anematode 10 minutes ago
Neither does the Collatz conjecture, Fermat's last theorem, ....

(Of course, those problems are on another plane than this one.)

reply
Eufrat 4 minutes ago
But that’s exactly my point.

These are absolutely worth studying, but being what they are, nobody should be dumping massive amounts of money on them. I would not find it persuasive if researchers used LLMs to solve the Collatz conjecture or finally decode Etruscan. These are extremely valuable, but it is unlikely to be worth it for an LLM just grinding tokens like crazy to do it.

reply
anematode 47 seconds ago
Maybe... but I would love if 1% of the investment in AI were redirected to the mathematics education and professional research that would allow progress on any of these problems...
reply
inerte 6 minutes ago
I would question at $60k. At $100k is a steal.
reply
wizardforhire 13 minutes ago
WTF!?
reply