OpenAI disbands its robotics team as its general models start running robots
90 points by sshah03 3 hours ago | 26 comments

jacquesm 7 minutes ago
General models can run robots. They are very bad at it. It is much more efficient to have a stack of models with 'dumber' but faster models taking care of lower level functions and higher level ones that are much slower taking care of high level planning. Consciousness isn't super fast, but it is very smart. Your 'lizard brain' (to use an overstretched metaphor) and reflexive system are relatively stupid but far faster and condense a whole raft of inputs to something your consciousness can deal with. For robots it isn't all that different. Sensors and actuators require millisecond level responses, that's something you're not going to get out of an LLM on the other side of a communications link.
reply
yewenjie 2 hours ago
The Bitter Lesson comes for everything.
reply
glalonde 12 minutes ago
the bitter lesson assumes compute is cheap and abundant, which is not really true in most robotics contexts. Unless your datacenter is right next to the robot, and you are streaming real time sensors to it.
reply
AndrewKemendo 36 minutes ago
I’m flabbergasted that anyone disputes this anymore.
reply
winfredJa 2 hours ago
This looks like general cost cutting to boost some numbers before IPO.
reply
seanmcdirmid 2 hours ago
They didn't fire or lay them off, they just disbanded the team and moved the researchers to other teams, so how is it cutting costs?
reply
ImJasonH 2 hours ago
More wood behind fewer arrows
reply
lavendermon 57 minutes ago
Not necessarily cost cutting, just streamlining and focusing on a few core products that can make money. ChatGPT has leveraged its writing style into a product that many consumers choose for their basic AI needs. The company is using information picked up in those chats to personalize the outputs, build trust, and most importantly raise switching costs as the users won't want to lose their personalized helper and confidant.

I'm already seeing some of my family members use ChatGPT in this way, even going so far as enjoying some level of companionship during research and decisions in their lives. I believe this product will have fantastic pricing power and can make the ChatGPT IPO look very attractive.

reply
chis 17 minutes ago
lmao
reply
eikenberry 37 minutes ago
So they gave up on non-cloud connected robotics then? Or was that ever in their scope?
reply
throwaway63467 25 minutes ago
Who says you can’t run an LLM inside a robot if you have enough energy and hardware?
reply
jacquesm 10 minutes ago
It isn't even optional. Anything with a motor should have sufficient executive authority on board to decide on how to safely power down in case of trouble. Loss of communications is a major impediment to deploying robotics to the point that I think it should be outlawed to have robots that do not have all of their thinking done locally.

But I suspect that this will be one of those regulations written in blood.

reply
ph4rsikal 22 minutes ago
[dead]
reply
deadbabe 2 hours ago
Is there a primer on how to actually use LLMs to control robots? Seems very token inefficient if you have to stream commands at 60fps.
reply
tracerbulletx 2 hours ago
Robotics are generally layered systems with lower latency/realtime systems running literally on the servos and other embedded hardware that use servo feedback and direct sensor access to operate, then building up complexity with tiers of increasing policy, usually all capable of working without the higher order layers which can inject desired states into lower order rings.
reply
lukan 2 hours ago
The LLM's are basically used for the big picture. What is the situation. What is the goal and how to achieve it. Not direct control of all the servos.
reply
mr_toad 22 minutes ago
The headline is somewhat misleading, as is tradition.
reply
almostdigital 2 hours ago
I think the idea is to just have the LLMs write the control systems and RL on that
reply
ares623 2 hours ago
Depends on what your goals are. If your goal is to give investors erections, then re-using your existing trillion dollar model is efficient.
reply
Tadpole9181 50 minutes ago
Saved the NVIDIA paper below for reading material, but if I had to guess just as a SWE how I would do it: use hierarchal thinking.

At the top sits a SOTA model capable of seeing snapshots of what's happening and directing intent. Then smaller, more reactive models translate the intent down into actions, before coming to a purely deterministic layer that handles inverse kinematics.

Sorta like how I control my body. I usually don't think about my fingers going to every key as I type - I know what I want to type, then that signal goes to some autonomous "using a keyboard" system, which invokes my muscle memory, which actually moves my muscles.

And if I get burned while I do something, my actual thinking brain doesn't really have time to process it before I'm already moving away; because those smaller systems have much faster - but more rudimentary - feedback loops.

reply
andriy_koval 2 hours ago
they have decision models now
reply
brink 58 minutes ago
It's ironic that the people working to make others disposable were in fact, disposable.

AI is eating AI now. The value of code has collapsed, while the value of the software industry overall has not increased. Could AI be the snake that eats its own tail in the long-term?

reply
pixl97 50 minutes ago
"Make a general intelligence, get surprised when it generalized "

Now your part of snake eating it's own tail makes little sense, AI isn't going away, only bad business models.

reply
dude250711 13 minutes ago
"long-term vision" - not even mid-term? I hope Anthropic knows better...
reply
metalliqaz 19 minutes ago
money is tight
reply
jeffbee 28 minutes ago
I wonder if this means anything about OpenAI's robot research office in Richmond, CA.
reply
mequelkramer 48 minutes ago
[flagged]
reply