LLM Agents Play Pokemon Showdown
17 points by grabnero 3 hours ago | 10 comments
minimaxir 58 minutes ago
As a tangent, there has recently been drama with AI and Pokemon Showdown because someone trained a bot (using standard reinforcement learning + a rewrite of the Pokemon battle engine to Rust for max self-play parallelism) that is apparently good enough to get to the top of the leaderboard.
replygrabnero 54 minutes ago
Yeah I saw that too which was partly the inspo for this experiment. I feel like its kind of getting to a place like in chess where AI is getting better and while theres alot of drama right now, I feel like it'll be something like stockfish in the future
replyjfreds 50 minutes ago
Did you try out different reasoning levels? Wondering if messaging is basically a proxy for additional reasoning
replypgn 2 hours ago
what was the selection for the teams? random per round? how does it play against people in ladder?
replydoctorpangloss 2 hours ago
Pokémon Showdown isn't a game so much as it is competitive Wiki reading.
replydeadbabe 2 hours ago
Jev will give better results on this kind of thing.
replynico 2 hours ago
For sure are many better models for this use case[0]
replyAt the same time, it’s interesting seeing how generally capable language models/LLMs/agents/harnesses are getting
We’ll probably see a lot more experiments and benchmarks like these in the near future
0: simple classifiers can play quite a few game, including Doom https://playground.jeffyclassify.com/#doom