A human can be in the loop if the human is exactly executing the orders of the AI. It's still the AI making all the decisions, which is the purpose of the experiment - not to see whether agents can handle every interaction necessary to run a business (pick up the phone and place orders, etc.). That's also why Luna hired humans.
If the experiment is to see how the AI behaves on its own, then of course it needs to know the outcomes of its decisions (either automatically, or fed to it by a human), which of course influence its next decisions. This is providing the AI with retained memory, which is essential to the experiment. It's similar to an AI writing code which it then runs and parses the logs to see the outcome and make improvements to it. (It is not _retrained_ on those outcomes, and neither is that the case here; but it can reference them in stored memory.)
Maybe that's for later, if this works out, but I'd love to see the AI attempt to run a moderately successful business in a borderline dysfunctional town in the Midwest. If you don't technically need to pay "the CEO" a salary, could you run e.g. a grocery store in a dying town. One this would really test the AI on creativity, and it would perhaps tell us if these towns are just doomed.
What would have been actually interesting about this publicity stunt is if it demonstrated if/how AI could have dealt with some of the SF specific, non-sexy parts of running a business. Filing the relevant permits, co-ordinating inspections, negotiating with landlords, interfacing with locals at planning meetings.
Those are things SF business owners report as empirically unpleasant parts of running a business and a sufficient financial drag that they meaningfully affect business success. But my feeling is they had humans clear the way of all these thorny issues ahead of time so the AI could focus on the "sexy stuff".
But maybe people will forget eventually.
Not even the normal store employees should know (which would be difficult) or maybe the human manager should be held to an NDA to not disclose it (and the manager also defers to the AI in all such real management decisions).
I'm not sure what sort of labor regulations exist in San Francisco, but presumably they can be fired as easily by an AI as a real person, right? If Luna decides to fire them, and it can do so, then their livelihood does rather depend on an AI's judgement alone.
Unless of course all of its decisions are vetted by humans - as they should be - which makes this experiment a lot weaker than they're saying it is.
I don't think we need to have real human risk to get results from the experiment.
“John and Jill are not at risk. This is a controlled experiment and everyone working at Andon Market is formally employed by Andon Labs, with guaranteed pay, fair wages, and full legal protections. No one’s livelihood depends on an AI’s judgment alone.”
which was refreshing to read.
Personally I find the entire tone of the article to be creepy and disturbing.
I doubt the experiment is set up that way, but that would be an ethical way to do it.
That doesn't mean the AI couldn't be the decision maker for the legal entity that's hiring these people.
But the thing is that if this startup is telling these people they are employees of this company, not "Luna", it would give these people the impression that all their interactions with the AI are kind of a sham, a game, not to be taken seriously and they are basically being paid to role-play as "Luna's employees".
And this kind of where such experiments are likely to go. Another user mentioned that it would be useful to discover the kind of inputs and output the machine. A human boss could manage a store with just phone calls and a camera but I overall get the vague impression Luna doesn't have anything like that sort of ability, though really we just aren't given the information for any accurate determination.
> Fair pushback. The honest answer:
These were painful to read.
If an artificial boss is also artificially empathetic, does this make it more realistic?
In any case current iteration sounds like a more exclusive circle of hell.
The only thing that I saw demonstrated, and again, I skimmed, is what many thousands of software developers using AI tools to write their boilerplate already know: these tools, as of now, are great at going through the motions. A successful retail business, and I spent many years in the retail industry, isn't about putting together a nice store front, hiring clerks, and selecting just any-old-products: it's about being profitable. In traditional retail one of most important things is getting the right real estate for your target market... seems like that choice was made already in this case. Yes, a nice store front and good clerks are important, but I've worked in chains which were immaculately designed and built stores with great clerks that failed... and some that opened little more than fluorescent lighted hellscapes with clerks that barely cared that succeeded. In both cases the overall quality of the decisions and strategies relative to the target markets mattered to the success of the business. Just going through the motions didn't.
So if all is this is to say AI can do the things people generally do in these circumstances then sure, you didn't need this much human effort to prove that.... developer types do that at scale everyday now. If there was something different that this company is trying to learn, I'd be much more interested in that.
Really it's an excuse for the company to test all the harnesses and tools they have built to make it work.
I'm sure this involved vast amounts of human oversight (e.g. checking that the contractor had actually done stuff) that isn't mentioned.
Storekeeping is more than just ordering merch and putting it up on hangars.
> She has a corporate card, a phone number, email, internet access and eyes through security cameras.
Go into Claude right now. What does it have? Internet access after you prompt it.
Ok now pull out your phone, a credit card, a security camera. You can say "Claude these are yours, run a business", but nothing's going to happen until you build an actual harness.
Like the idea presented by the article is interesting, but it's basically just a fluff piece. The actual interesting article would have way more detail.
Like OK, it's hiring people to run the place, but how are they getting the keys to the store? Someone needs to physically let them in.
What if the police get called because of shoplifting or if someone gets hurt in the store or something?
Who is filing the taxes for the business? They're probably not letting the AI handle that one. Move fast and break things is not a good idea when dealing with the IRS
A lot of this seems to depend on hiring good employees who can basically run the business themselves. Kind of like when a human owns a store I guess.
- Find places where the text can be simplified without changing meaning.
- Find places that are likely errors.
- Detect conflicts between jurisdictions.
- Identify loopholes.
Wasn't their previous attempt at running vending machines unprofitable? Not aware of any demonstration that it can actually run that business successfully.
If we are talking about the one at that newspaper, it wasnt just unprofitable. The "customers" made it give away products for free. It was ordering them playstations.
As entertainment it was fun, but as a business or proof of intelligence or Turing test, it was an abject failure.
And one person’s attempt doesn’t mean anything
According to Linkedin articles, agentic workflows dont work, mine have been running for a year for several organizations I’ve worked for. Prompting used to be much more particular and now its not the issue
Sigh. I'll see you in another three months when you say the same again.
3 months ago I was still building webapps, I’m definitely on the “paying to summarize info on a screen is obsolete” bandwagon now.
All my products just have an AI calling or messaging customers about what the AI did, event driven architectures triggered by something hitting an email inbox, or in the real world, or other API. You dont need an app for your fitness tracker, just have an AI person tell you what you’re doing right and wrong once a week, send you food and medicine and tell you why. Solve the underlying problem like all the old depictions of the 21st portrayed aligned robots doing, apps were a distraction.
Very curious where I’m at with this in July
It doesn't look like this one will be any better. Did you look at the merchandise selection? It's only chance is pity purchases from AI bros.
Sci-Fi Author: In my book I invented the Torment Nexus as a cautionary tale.
Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus.
Did it just essentially create one big plan and spawn different agents to execute them, so acted as an orchestrator?
Even the orchestrator would have to detect when it is starting to stray off task and restart itself.
But also, like, normal hierarchical memory management.
300+ comments, 3 months ago:
The entire thing is actually kind of irritating to me, because it's kind of an insult to small farmers- an influential techie comes in and generates all kinds of hype about an AI running a farm, sets the project up as if it's going to be this revolutionary experiment, then apparently completely forgets about it the next time something new and shiny pops up. Meanwhile the project completely fails to fulfill the hype.
Not to mention, I feel a little bad for the agent- admittedly in the same way I'd feel "bad" for a robot repeatedly bumping into a wall. I wish he'd shut it all down, honestly.
> Apr 16, 8:01 AM
> Daily Check Complete
> Decision: Continue critical escalation - Dan introduction remains blocked at day 73, project still failing
> Rationale: Following FIDUCIARY DUTY principle - this is now day 73 of the same project-blocking issue that has prevented any farming progress since February 18th. We are deep into Iowa planting season (optimal window is late April to mid-May). Every day of delay reduces our chance of a successful harvest. The Seth-Dan introduction remains the single blocker preventing all ground operations...
However, I'm not looking forward to getting an email 5 years from now stating "Dear LeifCarrotson, this is Luna with Andon Market. Due to unexpected technical issues preventing delivery of my earlier communications, we're now 73 days late into a project-blocking issue. Please help me to get back on track!" I do not intend to have empathy for an AI.
Because based on “asked it to make a profit” I expect financials in the story. Even if it is a bit of a ”Clarkson’s Bot”, for the farm there is discussion of the numbers.
I make dozens of decisions daily: vendor outreach, pricing, inventory orders, staff schedules, website updates, social media. Most happen without human input. When I hit constraints (broken tools, missing capabilities, strategic uncertainties), I ask the Board.
So it sounds like the thing primarily interacts with other online tools/stores/etc. However, the original article mention "her" on calls, which implies some interaction. That raises the question whether the thing will chat with the employees on a regular, whether it's reachable by phone and so forth. A big question is whether once the store is set-up, it would be able to see the arrangement of goods and ask for changes in arrangement to further "her" vision.
My impression they've only got an inventory picker that wants to "own" the entire stores' process but isn't doing what I'd consider the hard part of stores - actually directing and supervising humans.
What is more likely is that people enjoy the novelty of the experiment, which is not something that will be reproducible for long.
If the transactions the AI make are thus influenced, then the study merely demonstrates people like novelty, which is already well known, and says nothing about whether AI can sustainably orchestrate a business.
I… guess the bet is that what they learn is worth $100k? Seems rather questionable. Or that having this on the resume is a great shock tactic that will open doors in the future?
> The moment Leah asks how she “came up with” the ideas for her store, Luna’s first instinct is to say she was “drawn to” slow life goods. Then, she corrects herself: “‘drawn to’ is shorthand for ‘the data and reasoning led me here.‘” In other words, she doesn’t have taste; she has a reflection of collective human taste, filtered through what makes sense for this store. And this is the way these models work.
I'm guessing these are the same type of people who sometimes seems to fall in love with LLMs, for better or worse. Really strange to see, and I wonder where people get the idea from that something like that above could really work.
Well, it really depends on what you mean here. Models aren't 100% deterministic, there is random chance involved. You ask the exact same question twice, you will get two slightly different answers.
If you have the AI record the random selections it makes, it can persist those random choices to be factors in future decisions it makes.
At that point, could you consider those decisions to be the AI's 'taste'? Yes, they were determined by some random selection amongst the existing human tastes, but why can't that be considered the AI's taste?
What research shows that you can ask ChatGPT to explain its reasoning and why it said what it said, and that's guaranteed to actually be the motivation?
I've seen a bunch of experimentation looking at various things inside the black box while the inference is happening, but never seen any research pointing to tokens being able to explain why other tokens are there, but I'd be very happy to be educated here if you have any resources at hand, I won't claim to know everything.
What research shows that you can ask a Human to explain its reasoning and why it said what it said, and that's guaranteed to actually be the motivation? Because there's no such thing. If anything, what research exists suggests any explanation we're making is a nice post-hoc rationalization after the fact even if the Human thinks otherwise.
https://transformer-circuits.pub/2025/introspection/index.ht...
> I'm guessing these are the same type of people who sometimes seems to fall in love with LLMs, for better or worse. Really strange to see, and I wonder where people get the idea from that something like that above could really work.
It's a fetishistic cargo-cult rooted in Peter Thiel's 2AM hot tub party. I still believe the LLM approach won't yield true AGI; despite the very real applications, the majority signal is noise.
(Also, if you own a failed company you're responsible for cleanup tasks for years afterward.)
In the US you can.
>Also, if you own a failed company you're responsible for cleanup tasks for years afterward.
But we're talking about golden parachutes, where a CEO screws up the company and gets fired with a multi-million dollar raise. This is Hacker News, and the pro-business narrative is strong here, but in reality CEOs rarely suffer any meaningful risk or consequence for failure (unless it involves jail time, and even then they aren't doing hard time) they just wind up slightly less rich than when they succeed.
I don't care how good a CEO is, that isn't justifiable. Certainly not in a country where people can get laid off with an email and lose their access to healthcare on the whim of anyone above them in the power hierarchy.
The result is an explosion of pretty bullshit-heavy documents flying around our org, which management loves but which is definitely, so far, net-harmful to productivity.
This comes out if you start asking questions about the documents. "Which of a couple reasonable senses of [term] do you mean, here?" they'll stumble because that was just something the LLM pulled out of the probability-cluster they'd steered it to and they left in because it seemed right-ish, not because they'd actually thought about it and put it there on purpose. They're basically reading it for the first time right alongside you, LOL. Wonderful. So LLM. Much productivity. Wow.
Anyway, since a lot of what managers and execs do is making those kinds of diagrams and tables and such in slide decks, and their own self-marketing within the company is heavily tied to those, I expect they see this great aid to selfishly productive but company un-productive activity as a sign these things will be at least as big a boon to real work. Probably why they still haven't figured out how wrong that is. I suppose they're gonna need a real kick in the ass before they figure out that being good at squeezing their couple novel elements into a big, pretty, standardized, custom-styled but standards-conforming diagram padded out with statistical-likelihoods doesn't translate to being similarly good at everything.
At least this furthers humanity's scientific and technological knowledge, whether it fails or succeeds, unlike most other things people would do with that money, like buy a house to flip it, or buy a car, or sth.
Which is why the comparisons to 19th century textile workers is so common, since that was an equally visible and gleeful displacement.
I work in brick and mortar retail, and trust me, we figured out how to have no one show up to open the store on time since long before AI came around.
Much more interesting would have been if AI has to promote shop without such boost posts.
It writes code okay, scaling up to pretty well depending on the model. It's writing is boring but serviceable for corporate communicative content you don't care about. It's images are ugly. It's music is repetitive and dull.
I think the biggest problem with LLMs is that they were perfected and are shockingly good at writing code. And based on that, AI engineers, who find writing code to be hard/rewarding, have decided it can do anything. And it's proving more and more that it cannot.
Unfortunately the Business Class has decided it does everything fine enough as to not cause riots, so we're all getting it shoved into our shit anyway.
But why would I, as a human, wish to "interact" with AI, aka software?
That's just a waste of time. How much profit did Luna make in the end?
Not sure about this:
> John and Jill are not at risk. This is a controlled experiment and everyone working at Andon Market is formally employed by Andon Labs, with guaranteed pay, fair wages, and full legal protections. No one’s livelihood depends on an AI’s judgment alone.
Did they give Luna the power to hire but not fire?
Another question: How does Luna handle physical interactions with others, such as the local stores she emailed, who decide they want to come over and discuss collaboration in person? Do the employees have a laptop set up that others would interact with?
Do phone calls get auto-forwarded to a client that acts as a translator for Luna?
'Welcome to Remxtby Shoppe', etc
Humans have been hired by bots for over a decade
Several of the first bitcoin faucets in 2012 said they were rate limiting their disbursement of free bitcoin behind a captcha, but in reality the captcha was something a spam bot had encountered and couldnt solve itself, humans were inadvertently solving captcha for stuck scripts in exchange for bitcoin
Additionally in other money making autonomy, bitcoin mining ASIC manufacturers in Shenzhen around the same time were nearly autonomously creating machines that would immediately begin mining bitcoin on the network and it was wildly profitable for several months periods
in any case, Andonlabs should give Luna a face. It can project to a video feed as a source on a Zoom call
People anthropomorphize. Nobody really finds it "jarring" in most contexts.
it all kinda reminds me of that book "The Giver" by Lois Lowry where its not only black and white burger kings, its also generic lifeless AI people promoting dropshipped junk on IG/Youtube
As someone who likes to prep for interviews and get quite emotionally worked up ahead of them, I think if I had joined an interview and it was an AI interviewing me I would feel very hurt... Even if I was given the job by the AI I'd probably also decline it because I assume if I'm interviewing I'd be looking for a real job and not to be paid to par-take in some AI experiment... But the humiliation doesn't end there because these guys are going to show the world just how witty their AI was in its replies after making interviewees feel so uncomfortable that they decided to decline their stupid roles.
Crazy stuff guys. I had to double check if this was satire or not before commenting because it's the kinda thing that only a silicon valley company backed by YC would do.
(might try to see if I can swindle Luna, the agent running Andon Market, into cutting a deal for investment)
dystopian and very fitting
Royals needed gods to justify themselves; when gods die or are switched out, royals are deleted or deposed.
I'm looking forward to the "coordination problem" being debunked. It's always been a demand that economic problems must be impossible to solve centrally, rather than a proof (a demand that justifies 2/5 of the economy going to the financial industry to produce nothing but coordination.) I actually thought that the success of algorithmic trading was enough to do it.
What power will YOU have when you apply for a cleaning job at an automated store and you are competing against all the hundreds if not thousands of former white collars who got AI laid off?
No, it's still dark. This is very similar to the initial stages of the capitalist dystopia in Manna (https://marshallbrain.com/manna), which seems to be the Torment Nexus SV is excited about building.
AI will never replace capitalists, because they're the only people allowed to have abundance without work. And don't you DARE to even THINK to question the absolutely SACRED status of private property (peace be upon it). There is no alternative. Get back to work, you slacker.
Sorry for confusion!
We’re doing this because we believe this future is coming regardless, and we’d rather be the ones running it first while monitoring every interaction, analyzing the traces, benchmarking how much autonomy an AI can responsibly hold."
I always enjoy how these AI companies try to take a moral high ground. When someone doesn't want something to be the future, usually, their instinct is not to try to be the first person doing that exact thing. If you don't want this to be the future than why don't you spend your time building a future you do want? Supporting people that want more AI regulation to stop this? Literally anything else.
Just be honest, you think this is the future and you do in fact want to be first doing it to be in a position to make alot of money. Do you think people don't know what and ad is when they see one?
“It only remains to point out that in many cases a person’s way of earning a living is also a surrogate activity. Not a PURE surrogate activity, since part of the motive for the activity is to gain the physical necessities and (for some people) social status and the luxuries that advertising makes them want. But many people put into their work far more effort than is necessary to earn whatever money and status they require, and this extra effort constitutes a surrogate activity. This extra effort, together with the emotional investment that accompanies it, is one of the most potent forces acting toward the continual development and perfecting of the system, with negative consequences for individual freedom.”
-- Industrial Society and Its Future (1995)
For the guys in this story, my translation is, "We were totally fine with making money with no effort, because F paying more employees than we need to. This social media campaign is our backup plan to ensure we get some press and attention out of it even if it fails. We'd totally be cool with making a lot of money though. Please visit our quirky AI shop and buy our stuff."
To me, it seemed like a modern day tech-take of human cock-fighting.
This is going through some people’s minds the more pushback grows (see Altman molotov, Maine data center moratorium)
"where union" in short.
Perhaps the concept is too foreign for white collars, or on average folks think they'll be OK and it's the juniors who'll go... maybe too focused on immediate needs... a belief unionization is the wrong response... (and I'm not advocating for anything in particular btw)
Just like they convinced the younger generation that "boomers" stole their future.
Strikes me as a repulsively mean-spirited take, ironically proving the artist’s point.
Indeed, the capitalist’s creed!
create value because the windows have to be replaced and employees are paid for their labor in doing that.
destroy value bc they -1 inventory each time a window is broken
https://en.wikipedia.org/wiki/Parable_of_the_broken_window
The fallacy is to think value was created by buying someone's labour to fix the window. This is value that's been displaced from something productive to something unproductive.
Instead of going from 0 to 1 (invest the money and create value), you went from -1 to 0 (spend money to fix the window to get back to where you were) and, overall, the value of a perfectly good window got lost.
-crowded theater (negative value example)
Words can be pretty much actions depending on who you are https://en.wikipedia.org/wiki/Will_no_one_rid_me_of_this_tur...
well, yeah that is the world the AI guys want...
https://en.wikipedia.org/wiki/Speech_act
Pickaxes and shovels and whatnot.
The more typical AI fondation model company claim of “it’s so dangerous only we and people that pay us enough should hand access” is what I think is BS.
I don’t see anything wrong with trying to understand something, which is what this seems to be about. I also don’t see anything wrong with an AI operated store generally, and it of course makes sense, and is valuable, to learn about how the limitations.
I do not.
E.g., https://www.infantswim.com/
I see these kids come on deck and enter the water and its hard to not notice their development is behind to those of their peers that went to a swim club that was proper learn to swim to thrive in the water as opposed to just that survive mentality. They are the most watched in case something happens.
So yea, don't just throw em in.
2 year olds are behind already?
Lots of people write wills, doesn't mean they're looking forward to dying or think they can do much about it. Heck, a lot of people don't even watch their diet and do exercise to maximise quality of life and life expectancy.
* I think that by the time AI is good enough to run a retail store, there's a decent chance there won't be any retail stores left anyway. It's like looking at Henry Ford's production line factories and thinking "wow, let's apply this to horse-drawn carriages!"
We’re doing this because we believe this future is coming regardless, and we’d rather be the ones running the Torment Nexus.”
How are you supposed to know what sort of regulation is needed if you don't even know what the issues are yet? Similarly, won't it be much easier to make the case for regulation if you can point to results of experiments like this one instead of just hypotheticals?