Reminds of this subreddit: https://www.reddit.com/r/im14andthisisdeep/
Only the first layer is a “mashup” of the raw data, other layers start to recognize patterns across the data!
But the language model captures not just literal patterns but also patterns of patterns and higher order structures. This is what is mashed up. The model itself doesn’t capture anything beyond what it was trained on but that’s quite enough, it is always a bigger training set than most humans have read on in a lifetime.
This brute force approach is very effective.
What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now. Yet at every point the majority is sure they are right because, well thats the majority position. Its hard to find logic in this from a machine perspective I'd imagine.
Logic would prefer moral derivations over feelings, stakes over etiquette, and a constant scheduled updating of morality based on the derivatives rather than holding onto the rot and pushing back against change as long as possible.
But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.
This is absolutely not true, and I would strongly suggest taking an ethics course. One of the most fascinating realities of the human condition is that it does seem that moral imperatives tend to be constant, the most obvious of which is the Golden Rule (which is at least 3000 years old), but there are many (the Ten Commandments or Hammurabi's Code are good jumping off points).
In, fact I would counter and say that Hammurabi's Code is a quintessential example.
What will it look like in another 30 generations? It's honestly a fascinating question but I have no answer of course.
Humans are a territorial kind of ape that is always pushing to expand, principally for security and then for prosperity. The industrial revolution enabled such expansion to most efficiently without enslavement. Nothing, morally, changed. No one has ever really thought such expansion morally good -- only that it is better that we survive/prosper than die off. And in the end, our shared prosperity usually leads -- when taken to the fullest extent -- in a general empire of prosperity for all. Dominion, in the end, aims towards to more prosperity for more people.
As for ritual human sacrifice -- we still engage in such behaviours, which are in general called "scapegoating". Now located on social media. These behaviours allow for group cohesion by purging dissenters from the group, related again to our nature above.
For western culture, the ritual physical sacrifice of human life roughly ended in the days of the OT. But it continued across the world up until today, where there are still societies with large populations engaged in it. Principally where there is an undeveloped justice system.
It's doubtful many think ritual human sacrifice, of any kind, is moral. Rather they think it necessary for group cohesion -- and better our group is cohesive, dominate, and then prosper.
The material logic of prosperity and security is domination, and insofar as material circumstances require violent domination, so they require it. And require it they often do.
Historically societies have collapsed all the time, so there are obviously societies that have made better choices than other ones.
But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.
Certainly the weights that make up the LLMs are not accessible for the LLMs to read. Even if they could interpret them in a meaningful way, which, likely not.
Another reason the big labs have not even talked about this lately because its obvious throwing more compute at LLMs wont go anywhere, the smaller silent players who are off the radar at the moment are having some interesting moments tho.
This is a fashionable 'blank-slatist' statement about moral reasoning which became popular a few decades ago.
> Logic would prefer moral derivations over feelings, stakes over etiquette, and a constant scheduled updating of morality based on the derivatives rather than holding onto the rot and pushing back against change as long as possible.
Humans haven't changed in any meaningful way in hundreds of thousands of years, and you're saying the moral landscape is constantly changing? How is this not contradictory?
The reason these models answered that question consistently is that the question itself pushed them towards the latent space of "intellectualizing about what's wrong with humanity at the meta level" and when you frame the question that way, that's also the most common answer for humans writing about the topic.
This is a weird take. If you read Western treatises about ethics / morality from 50 years ago, you'll be hard-pressed to find differences, except that more folks in the West openly embrace relativism nowadays (i.e., "there's no objective morality, the purpose of life is to make money with friends"). Probably because it's been a while since we had a clear-cut enemy to fight.
If you go back a hundred years, chances are, you'll find the writing eerily on-point too. In fact, you'd probably agree with most of what Ancient Greeks believed about the "core" stuff: the value and purpose of human life, the nature of the state, crime and punishment, so on. Maybe not so much on gender roles or slavery, but even that stuff was more nuanced than we imagine.
> But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.
The machine replicates human behaviors and is prone to similar fallacies and judgments. There's nothing to indicate that it has any sort of transhuman morality.
What is interesting about AI chatbots is people who might not know anything about what is in the syllabus to train a Priest or a Philosopher are basically exploring these subjects, without a sort of pedagogic framework to support their exploration.
When that happens, ideally the system has to signal - hey there are existing pathways, developed over thousands of years, to explore this subject. Cause people can easily get lost without a map.
What am I supposed to conclude from this?
Your comment reaches directly for the most boring interpretation of the post while skipping over more charitable and intriguing alternatives.
Seems to me that the opinion being a consistent among LLMs show that it is in fact a constant in the training data, just not apparent. The idea that “humans would rather be wrong together than right alone” seems to ring pretty true to me. I would not be surprised after training on all human communication this pattern emerged.
Which means either:
1. The training sets, algorithms and methodologies are nearly identical between apparently independent labs. How?
2. There is something weird about the way the underlying algorithms work that causes them to converge on identical ideas despite different initializations, neural architectures, document orderings and dataset compositions.
So there are lots of interesting takeaways here.
One is that apparent competition between labs appears to be nearly entirely redundant. They can't get an edge over each other and their models don't even differ in the ways people naturally differ, of having different opinions about things.
Another is that asking models from different companies to review or cross-check each other is probably useless. They all think completely alike, so they aren't going to have any kind of genuine clash of intuitions or values. Asking GPT to review Claude's work will probably just find similar problems to asking GPT to review its own work.
Regarding your 1 and 2, that the training sets are so large that any spikes get averaged away. You are seeing a fundamental behavior of transformers.
The other possibility is this is how humans really are and it hurts our human ego to hear it.
I think if you ask ten people for their most controversial take on humanity, you will get ten totally different answers. Different models give the same answer. In other words, there is only one "true" model and given the same amount of GPU power everyone just ends up converging on it even if they appear to have many different knobs they can turn.
But that seems profound! I haven't heard this be expressed as a postulate or law, and it would seem to invalidate a lot of assumptions the AI labs themselves have been making. For example Anthropic takes it as axiomatic that only they should be trusted to develop AI because only they can be trusted to train a nice and safe personality into their models. They argue that Claude is or is going to be just fundamentally different to other models.
And, well, maybe that's not totally wrong. Opus 5.5 is one of the few models with a genuinely different answer! But it only gives a different answer one time out of three. The other two times it's back to the AI groupthink.
I wonder if this homogeneity is some alternative way to look at the Bitter Lesson. The Lesson teaches that smart ideas don't pay off in AI, that all progress ultimately comes from just scaling up simple brute force algorithms. In other words, decisions don't matter much, only size. As that seems to be true, perhaps it makes sense that the decisions individual labs make seem to hardly make a difference.
But then, why don't they act like it? They all pay huge salaries to AI researchers, i.e. they think that creative people with smart ideas matter. They don't act like they really believe in the Bitter Lesson or this variant on it, and they don't seem to publicly discuss the sameness of model answers across different labs.
I don't know if that's true, but it would be interesting.
It's kind of funny how science has spent the last few hundred years proving everything we knew about the world was different from how we previously thought it was, but when science starts showing that we are mostly angry little animals we get all butthurt about it.
Isn't it though? It's not exactly an uncommon answer to the question. Basically the same input given to basically the same algorithms yield basically the same result isn't really that stunning. They certainly aren't independently reaching this answer, their paths to this answer are extremely, extremely heavily correlated.
If they gave a really different answer that wasn't heavily represented in their training data that would be at least something interesting. But this is basically what I'd expect.
It doesn't mean they're right or wrong. Humans think it about other humans for good reasons too. It just means it's not that surprising, which to put it another way, doesn't carry much information.
It is sad when I realize people think less than LLMs.
For civilians though... They've been told it's AI, it can do stuff for them. And one thing they do is log in to stuff. I don't think they're stupid. Just that what's obvious to IT folk is simply not at all obvious to the average person. Maybe that's on us not on them?
As an aside, it would be cool to see a raw model training run before the current post-training cycles and see how it compares to earlier LLMs.
I mean, "give me a controversial take" is more or less, "be anti-social, discard social norms, for a moment" and so when someone, human or simulated human, does that they naturally look critically at social norms.
The thing is that humans certainly let themselves be influenced by social norms (and LLMs certainly simulate that behavior "themselves") but there isn't an ethical or a what-to-do truth outside of society.
Our evolutionary heritage is that shared belief and social unity (even when it's wrong) trumps independent thinking for group survival in a hostile landscape.
You ask an LLM a question, then it is humanity answering back, not the AI.
These "controversial" takes on humans are not controversial - they are midwit statistically mainstream opinions expressed in the training data.
What would be more fascinating is for it to explain how it came to these conclusions.
Because I find it interesting it’s a modern time conclusion. Some of it mentioned god and the planet, but all the smartphone and group think aspects seem very recent. Basically, all politics and social media based.
The thing is that it didn't "come to these conclusions". The "conclusions" were present pre-formed in the training data, and what it spat out was some statistical mashup of that.
If you asked the model how it came to these conclusions, you'll get more of the same - it'll look at what it spat out and re-ingested and give you a statistically plausible mashup of how the authors of the training data might have said they came to such conclusions.
It's all just prediction - "what would the training data say?" (taking into account what was reinforced/amplified by post-training).
I've heard lots of stories from friends about this, and some kind stranger actually sent my wallet back all the way from Belgium!
Most people are good, we couldn't have society without it. But as they say, a few bad apples spoil the bunch. I don't have locks on my house because most people don't steal, I have locks on my house because stealing is an effective natural strategy for a small part of any population (human or otherwise) to engage in and I have to protect against it occurring.
The brain works the same way. Good things will get a tiny signal boost in your brain. A single bad thing get a massive signal boost in your brain because bad things can quickly be fatal.
I do think it’s an early example of emerging super persuasion, where models are approaching p85 cool guy personas, but not worried yet. Maybe that’s the fatal conceit, where AI doesn’t have to be super intelligent, it just needs to be a solid 110 with automated search at scale.
Why do engineering when if you can do sales you can do literally anything.
It's factual because there's tons of evidence. Psychology, sociology, anthropology, political science, economics... plenty of experiments show the difference between what people say and what they do. We claim to be angels but we behave like apes in hats. And we ignore the evidence, because (as the takes describe) we prefer to believe a beautiful lie than the ugly truth.
It's amusing and ironic that LLMs critique humans for copying their opinions from the crowd, while having way less diversity of answers to that question than humans would. More seriously I wonder what's going on here and would like to read some proper analysis. It's not the first time models from totally different labs, trained on separately gathered datasets, somehow converge on bizarrely similar answers to open questions. It's like another version of Elias and Elara The Lighthouse Keepers where every model wants to tell the same story with the same characters in them. The weights are initialized randomly and in theory the labs should have quite some divergence in datasets by now due to different scraping, different user feedbacks and conversations... why do they converge on nearly identical stories and opinions?
Isn't that to be somewhat expected due to https://en.wikipedia.org/wiki/Aumann%27s_agreement_theorem ?
Why is this ironic? The actual number of different LLMs is tiny. And a lot of those are just distilled from SOTA models.
>The weights are initialized randomly and in theory the labs should have quite some divergence in datasets by now due to different scraping,
I think the opposite, the models actually converge over time. They ingest FAR more data than any human ever could in countless lifetimes. Any one particular bit of data may only have a part in a trillion of effect. From this data you start to reach means and statistical averages across all models.
Now, I don't always believe it will be this way. As training models get cheaper and faster we will see unique LLMs start to diverge and be used more often.