I know this sounds unusual, but I do think this wouldn't be the worst thing that could happen. Sure, it would cause a bit of chaos, but it might also make us think twice about continuing down our current path of unfettered development, and possibly even make us much more cautious, which might in turn even help us with the climate crisis.
Because human beings in a large mass only respond to one thing: immediate difficulty. They don't respond to delayed difficulty like "the climate is getting worse" or "technology will make life more difficult in the distant future". They only respond to immediate pain.
So I think it would be a good thing if the financial system had a serious threat. Then people with money, who tend to offset future problems with technical solutions and financial padding, would start to become much more suspicious of technology and defund AI and wasteful things that are likely to threaten their safety.
Maybe the long term changes such blackouts would cause would be "worth it" in a utilitarian sense, but let's not hope for such damage.
Blockchain despite all its flaws, is the canary in the coal mine. They have been dealing with an increasing amount of cyberattacks. Because it all happens publicly in the blockchain, there is no way to hide it.
However major bank gets hacked in the tune of over 100 millions and we get to know it happened because somebody who has no relationship to the hack turned his eyes to a bunch of suspicious transactions happening on Ethereum.
This is why I think AI is so stupid. It's just too powerful for anyone to control, especially with the inertial mass of capitalistic society powering it through the incremental, short-term advantages it gives.
I disagree that something like this would be a good thing though. Whether it's a power outage, financial crisis, or something else people will die. If that's what happens and society ends up adjusting course after then that's what happens, but I can't call people dying a good thing.
This sort of thing was going to happen, with or without AI (state sponsored hackers, the US or China taking down systems they control, or whatever).
The later it happens, the more we will be down the path that lets it happen, and the worse the consequences. We are getting gradually worse consequences: AWS outages taking down websites, GCP outages taking down banks, attacks on fridges disrupting good supplies...
Its better for us to change path when something happens that causes dozens of people die rather than wait until its millions.
I don't think that's a good thing. But if we DON'T do anything, then even more poeple are likely to suffer. So while people dying is not a good thing, a thousand people dying is better than a million people dying because we don't do anything.
Same thing about burning your hand. It's not a good thing to burn your hand, but burning your hand once when you're young probably saves you from a more serious accident later on.
If someone manages with the help of AI to hack into a vital system, then the system was already in a critically bad state beforehand. If developers can't use AI to check the code for security flaws, then security gets worse and not better.
You’re placing blame on the wrong spot. It’s algorithms and social media incentive systems not AI. Our digital media incentives addictive minimum effort chum and slop. AI just makes it easier to make.
“I think [Mark] Zuckerberg and [Elon] Musk and Sam Altman, they understand the risks, and they don’t give a shit. They’d rather have the money. That’s my belief.”
I still don't know how Karpathy still works with those folks. I really think the guy is great but he makes me wonder
Programming has never been this awesome.
That's a touching sentiment or whatever but the nature of the thing is that it will inevitably be opened and that it cannot be closed thereafter. So (I think) we'd all be better off putting our efforts towards predicting the likely outcomes and figuring out how to make the best of them.
Looking back now, was it really happier before AI existed? I'm not sure. In the past, when I wanted to find out whether I was right or wrong about some piece of knowledge or information, I'd contact several people and, far from getting an answer, I'd often be told not to bother them. But AI, in that respect, is the best teacher I can think of.
I think the times are continuously getting better. It's just that there's a problem of power concentration among the people who handle it. Still, if AI is a tool of that nature, then I suppose that can happen.
But considering what access to knowledge was like before AI came along, I think having AI is better.
He would still be right and the field practioners would be wrong.
That's a relatively silly question, don't you think?
Because a certification has to be fixed at a particular point in time. Korea's certification exams start with the 23 GoF patterns and go on to rote memorization of countless patterns and layered terminology.
The problem is that the way you learn for this certification is the old way, so someone who gets the certification ends up writing legacy code.
But if you always update the questions to the latest, then nobody can pass the certification. So, having seen how the certification system works in East Asia, I don't think it's all that good
No way to tell until after, though. So far, mutually assured destruction is working. It works until it doesn’t.
Same with AI. Thus far we’re better off with it than without it. That may change.
I mean, look what we did with fossil fuels. Some people are even still willingly burning them, even though the data is clear that our use of them has already started the total destruction of our civilization.
Is there a name for this, a technology that is so insanely useful that there is no way we won’t exploit it, but where the worst case scenario is total collapse, and impossible to determine well in advance?
(Fossil fuels I suppose don’t count as even the energy companies knew unambiguously decades ago that large scale use would eventually bake the entire agricultural system out of existence.)
The reason for that was more that the 3-4 great empires involved in the previous two were exhausted, Japan was defeated, China was in no shape, and US vs USSR had no reason to push for one, since the Cold War era gave them plenty of breathing room, roughly half the world each.
... no shit Sherlock
TFA could have been two sentences:
He warns that AI might cause serious problems: power failures, job losses, and greater social inequality.
He believes AI might improve our lives, but governments must make sure it is safe.
ok boomer
Seriously? It seems to be causing more problems, giving us more boring work, and less free time.... sounds like absolute bullshit.
We need to build it as fast as possible because it could cure all disease. Ignore whether that's possible, how it might happen, whether all we're missing to do it is more intelligence, etc.
The only way I see us curing all disease is to throw out all the names we've given to collections of symptoms. What disease? I don't see no disease, you just have a sore throat, runny nose, and mild fever.
Hell no. We need to risk pumping out even more CO2, causing more environmental disaster, ruining entire human communities, just to cure diseases? You might as well say it's better to be in prison because you're guaranteed three meals a day. The logic of some pro-AI people is baffling.
The other day I hired an accountant and even he sent me an AI answer that sounded right but was actually bullshit.
In short, it takes a LOT more time just to find some sanity now.
AI is shit.
Only 14 years old and accelerationists think we can just push the capitalism pedal further and somehow get social changes going their way
The carelessness is also clear in how AI is rolled out. It's being presented as a disruptive technology, as is everything else presented by modern tech companies. The goal is a very large payout in a very short period of time, it's not to make anything better, or worse, either is just a side effect.
Had LLMs first been presented as a "better IntelliSense", where rather than auto-completing function names, your editor can now suggest implementations, very few people would have been upset. Same for writing, it's just a better version of Grammarly, it's fine, or as a tool in drawing software. The issue would have been, for the AI bosses, that this takes time and doesn't create trillion dollar companies. It only creates companies that they'd actually have to run, long term, and that's boring.
The AI companies are as much being pushed along by investors as they are in the driving seat, but the push is in the direction of getting real products in front of people and experimenting with this miraculous tech everywhere it could possible work.
How many, approximately?
Value is simply the mutual understanding that something should cost a certain amount, and the system we've built emphasizes the need for profit above all else. When a set of companies comes along with a new, impressive technology claiming that they can automate away the white collar workforce, they're trying to capture as much value as they can.
It's not a moral sickness, it's market actors behaving rationally. Believing that people should be willing to leave money on the table because it's the right thing to do is a dead end, the entire system needs a rethink.
No matter what practical benefits this technology might bring, if this runaway marketing campaign isn't stopped, in just a couple decades we'll see what the complete collapse of civil society looks like, a world where the institutions no longer see value in maintaining an educated democratic populous, and can relax back into the comfort of feudalism.
If it weren’t OAI and Anthropic, it would be others.
If it weren’t US companies, it would be others.
It is far too useful now to stop large scale deployment, it isn’t a matter of specific people or countries not being cautious or prudent enough. The entirety of the Earth is at issue now and there will always be people who cause the tragedy of the commons (if indeed AI turns out to be harmful in the long run, which is not remotely certain).
Look at what happened with fossil fuels. Even if the entire global west totally ceased use, there are many countries that simply will not stop because of local utility. AI is the exact same way.
There is something deeply rotten about the US mindset though, them being American is definitely a problem.
> Look at what happened with fossil fuels.
The west had a 100 years head start, on top of that these countries are producing everything you consume, don't start pointing fingers. Chinese current co2 per capita is the same as the US in 1900
Releasing artificial intelligence for the use of all of humanity is better than it being controlled by one or more shadow groups. That's what the Americans did, and the Chinese followed in.
Had AI been invented in the 20th century, it would have been kept secret and used only by state actors to facilitate genocides at mass scales. But we live in different times.
Uh? In what universe are openai and anthropic not shadow groups?
Doing what? Rushing to make trillion dollar companies, rather then developing the tech and their companies in a sustainable way? I don't even know what in the original comment this would be a response to.
Or have you forgotten that Sam Altman was ousted by individuals that shared your concerns, and it was the OpenAI employees that rebelled against the board and wanted Altman back?
That HN post was infamous and nearly took down this site.