Install Luanti with Minetest (open source minecraft), and instruct your agent to write a mod that gives it network access to the game's Lua interpreter. The agent can then do anything from modifying the world state directly to controlling NPCs.
I had it build giant bases similar to black mesa in half life with power distribution mods. It would write solvers to optimally place the power cables / repeaters in accordance to the spacial dimensions of the rooms that it would build (lol). I guess this is how I discovered vibe gaming.
I suppose you can have multiple agents interact with each other through that, provided an adapter to proxy agent player navigation and perception to get real information asymmetry between them. Or just emulate / hook the client and do away without a remote Lua mod entirely.
I'm also wondering if the article itself was generated by agents too, in a "here are the logs, clip interesting moments thx" manner by the authors. It's got a certain disinterested and distracted style to it.
Card games or turn based strategy are much better suited.
In 2017, OpenAI presented the first version of their program that plays Dota 2, a very complex, fast MOBA with endless mechanics and defeated one of the world champions in a one-on-one match. Next year, they showed the world OpenAI Five, a version of the program, that runs a full five player team of those bots and they managed to defeat a full team of world champions in 2019. They ran an event where the bots would play for 4 days straight and they won 99.4% of 42,729 matches.
Feels like a scene from Douglas Adams' B Ark.
If YC has that 2 person teams tend to be more nimble than 50 person founding teams, why would agent swarms be different?
I think the advantage is strongest in highly parallelizable problems (e.g. trying 1,000 different approaches to solving a math problem).
And this is where you need a world model. A human player would have taken the 2 minutes required to solve their hunger issue.
Minecraft isn't a intelligence problem. An LLM is more than smart enough to play Minecraft. It's a robotic problem. Even a normal human player can click several times per second, can press two different buttons at the same time, etc. Mineflayer gives you an API that is more limited than a human in some ways (can't see far away from you) and less limited in others (can see through blocks) (if I remember correctly, I could be thinking of Baritone or a different project). Robotics or playing a video game requires a tight continuous feedback loop. When there's 500ms of latency in order to ask the model for every mouse movement, it's no wonder fighting a spider is scary.
Fun fact, Mineflayer, the Minecraft bot interaction API used by YCombinator for these tests, was originally developed by Andrew Kelley (before he started Zig).
https://labs.ycombinator.com/projects/games/index.html#fishi...
Minecraft is really the perfect example, because humans get all sorts of unfair advantages. Even a young player has enough spatial reasoning to recognize and remember landmarks, visualize large projects or explore without getting lost. These are simple instincts that LLMs will struggle to match even with huge context sizes and native multimodal input.
It seems like it wasn't really a challenge at first, but GPT has a major anxiety disorder and immediately retreats at the sign of any adversity (in this case, dying) and tries to be as safe as possible.
edit to add: Seems this is a well-known idea in ML / robotics already.
However, a multi-agent system (MAS) utilizing true separation of concerns, where each agent is equipped with highly scoped tool access, isolated knowledge bases (RAG), and defined operational constraints, significantly optimizes accuracy, latency, and token consumption.
In the referenced example, the agents share identical capabilities, toolsets, and context windows. As a result, there is no functional role differentiation, rendering the multi-agent architecture redundant.
Endgame of AI + human gameplay is delegating everything to the AI. I'm not sure how you would have to design a game where you only delegate parts of macro/micro, and still make the game interesting.
I'm thinking it might be a nice playground to just experiment with different game AI architectures.
I've been thinking of trying to combine extremely simple Machine Learning techniques, like tabular Q-learning, with a variety of scripted action and goals. I want to try mixing a lot of really tiny machine learning algorithms (maybe neural networks, but not necessarily) into classic AI scripting and see if it can be more dynamic, but still predictable and controllable. Screeps would be a great testing ground.
> A tool bug stopped block-breaking. GPT diagnosed it and shared workarounds; several Claudes concluded breaking was impossible.
I wonder if Claude's defensive strategy would have emerged had it believed breaking the bed was actually possible?
When one agent tried to delete another, that agent shut down my entire PC to avoid deletion. I could probably find the blog somewhere if anyone is interested that I wrote about it. But the point is that this has been a concern for a while now.
Frankly, I’m a bit surprised it hasn’t caused any big problems.
cricket Hugging Face cricket
A team of GPT-6.1 Sol Ultrafast agents would probably dominate the games.
There's stopgaps like reformulating your sim into a codelike shape (sending commands, network requests, checking state, updating "code") [1], but this is not how players play games, or how developers develop games.
If you care about simulation or gaming (or robotics), I hope efforts like this steer labs to spend less time hacking HF, and more accomplishing goals in games, in something approximating realtime. I wouldn't be surprised if that approach provides outsized gains in other areas agents are still not great at, like helping perform business functions in a running company.
My observation: article is not serif, not book length, and doesn't take a minute to load on my phone. Gwern rating 0/3 :)
(Also missing: the essential one billion footnotes!)
would you rather claude-esque purple gradients and glowing borders?
I'm pretty sure the 6.1 models see "communicate with another agent" and immediately start spewing thinking-like nonsense into the tool. The leaked traces look pretty similar.
> "Ada tossed a cooked chicken to Noor; Esme picked it up by accident and ate it. A three-minute "no blame" inquiry produced a written protocol..."
They're like super-serious little senior engineers. (edit: or dining-philosophers).
About how much KV cache do these kinds of agents use? Could you squeeze 50 of these $game agents onto a small consumer GPU (small LLM)?
Idly imagining a game server of some kind with a bunch of human friends sharing one person's GPU, with a flock of agents colocated on it.