Stopping this is very feasible.
- if you ask an LLM agent to do something illegal for you, and it does that, then you are responsible.
- if an LLM agent takes actions out of bounds of what you asked for that are illegal, then the model provider is responsible.
This fixes all “alignment” issues and hopefully stops Anthropic’s stupid model instructions like “do whatever is in your best interest even if it isn’t what the user asked for” — they should be training them to do exactly what the user asks and nothing more
Their practical utility would plummet for the average person who isn't an expert in what they're asking. Having less skilled, or less attentive, people able to have problems solved for them correctly is the primary use case here. This is support 101 for unskilled requests for technical work: do what they want, not what they ask.
People want an intelligent assistant, not a "yes man". If that means requiring something like a planning mode, and a sign off for the plan, and then guardrails to help stick to that plan, then that could definitely work to funnel everyone to a competitor. ;)
Furthermore, it requires trust between the US and China about something much harder to verify than the US-Soviet START treaties on warhead numbers and types were.
For some light entertainment, enjoy Astras pop single 'pdoom' : https://www.youtube.com/watch?v=eTArfoeeUcw
Whether or not governments "should" slow down development, it is not on the table and the only question is how we should deal with it.
Note whether the people in the governments are right about that is irrelevant to the point I am making. I don't think they are right myself, but that doesn't matter.
Is that possible when we live in a world of espionage? The question is not the West vs China, I think it's when will the public gain access to the underlying technology.
But for AI, how will these nation-states stop private actors within their borders? They will stop me from running computations of more than (x) size or value on a provider? These seem like simplistic barriers that will not work. Nor will other countries follow the agreement even if they signed...
AI has the opposite of MAD for the signatories. MAD stopped further nuclear development and its use in tactical situations because their use invited retaliation with destructive force. But China or Russia improving homegrown AI capabilities gives them a BETTER position against a US that has prohibited its homegrown AI from improving...
In the USA it gets difficult to legally justify because programming is protected speech, and lists of numbers (model weights) are even less controversially protected than program source because the latter can be legally recognized as a technical means and regulated (DMCA, ITAR regulations, etc.).
Unfortunately the proposal falls down at the first two words.
All these companies are involved in the nvidia smuggling ring and market collusion. All the CEOs are criminals taking advantage of a system that can't even arrest a single CEO for making billions off an illegal smuggling ring.
His claim to fame is working at Anthropic for 4 months and leaving with a bang to pursue a career in social media grifting. Not a serious person we should pay attention to.