I am a great fan of demoscene (computer art subculture) since middle school, and hence it was a ritual i had to perform.
For estimating the Elo, I measured 240 automated games against Stockfish Elo levels (1320 to 1600) under fixed depth-5 and some constrained rules, using equal color distribution.
Then converted pooled win/draw/loss scores to Elo through some standard logistic formula with binomial 95% confidence interval.
Bug report:
a b c d e f g h
8 r n b q k b n r 8
7 . . p p p p p p 7
6 . p . . . . . . 6
5 p . . . . . . . 5
4 P . . P P . . . 4
3 . . . . . . . . 3
2 . P P . . P P P 2
1 R N B Q K B N R 1
a b c d e f g h
move: b2b3
ai: b6b4
Appreciate you taking the time to test it.
[2] https://www.chessprogramming.org/Sequential_Probability_Rati...
I'd claim it would work on human opponents, but I think it would get banned from chess tournaments.
The problem isn't really well defined. Elo rating is assumed to be determinable independent of what opponents you face, so scoring 50% against opponents rated 1800 gives you the same information as scoring 26% against opponents rated 2000. In practice that's obviously not completely true, and for degenerate examples like the ones we are discussing it completely falls apart.
> Not implemented: castling, en passant, promotion, repetition, 50-move rule.
If anyone reading this is interested in small and efficient chess programs that are still reasonably strong, there was a x86 assembly port of Stockfish called asmFish from a couple of years ago (the Win64 release binary was about 130KiB). Also see OliThink (~1000 LOC) and Xiphos which has some of the simplest C code for an engine of its strength that I have seen. I have not investigated the supposedly 4K sized engines that participated in TCEC too closely but from what I have seen so far it would seem that there are a few asterisks to be attached to those claims.
[1] https://github.com/thomasmueller/bau-lang/blob/main/src/test...
I understand this may not sound very convincing yet... it is hard to describe... basically, unsigned types sound simple, but they are not.
so every game stayed in the same no castling variant
and you're right, this rating is for that constrained variant, not full chess.
For castling you should be able to fix this by specifying that castling rights have been lost in the positions you give it.
For the others I think you would need to filter not just when giving it the list of allowed first moves. You'd also have to make it not consider en passant and promotion in the search, by modifying its move generation.
The static evaluation would still be off a bit, but that probably would not have much effect most of the time.
From what I've read it is feasible to train a new neural net on a decent home computer in maybe around a week, but that's probably overkill for your use of figuring out how strong your engine is at no castling/no promotion/no en passant chess.
Do you work with it like this or do you have some sort of script you apply to get it down to a single line, single letter variable names?
also removed castling/EP rights from FEN
It is hideous now!
P.S. I assume 1200 elo in chess com scale (not lichess / fide elo) and bullet chess variant?
These things always make me think back to Westworld season 2, where the finale revealed that human minds are much simpler than they themselves believe and fit completely into an algorithm that could be printed in an average book.
So while 4k is still very impressive for the code base, it comes with a significantly larger runtime footprint.
According to TCEC the time control is 30 mins + 3 sec, that’s a lot of compute!
> It's wild to think that 4096 bytes are sufficient to play chess on a level beyond anything humans ever achieved.
Now if you had a very good chess program running in very constrained (dynamic/RAM) memory, then that'd be partially more revealing. From a cursory search there's a 1800 ELO engine for the C64, which seems very impressive but very far from the best human players.
I'd be interested to see a curve of ELO x Avaliable RAM for the best chess engines (up to given RAM), and how that compares to other games and activities.
On RAM vs ROM (program size) memory, I think at a high level dynamic memory helps you keep track of search paths in a large tree search, saving you some computation. Program size tends to enable improving the effectiveness of your search heuristic, as well as pre-computing e.g. initial and final game optimal moves (potentially saving arbitrarily much compute). I like thinking about those things because I think the search paradigm is pretty informative of computation (and even intelligence) in general. Almost every problem is basically some kind of heuristic search in some kind of space. And you tend to get better at things by refining your heuristics (usually through some experimental training process or theoretical insight), considering more options, exploring deeper consequences, etc..
I think what really defines humans isn't really our ability to solve problems or play chess well etc. (although that's extremely useful and also enjoyable most of the time), it's really our emotions and inner world. We are not really Thinking Machines in essence, we're Feeling Machines most significantly. The thinking part is a neat instrumental part :) We can delegate thinking to machines but what we cannot extinguish is feeling or the human "soul", because that is the source of all meaning.
(Partial answer, 2kB is a very small fraction of what we'd like to think counts as human.)
2kB is a very small fraction of what we'd like to think counts as human
This doesn't seem to mean anything. Why would 2KB have any relation to "counting as human". It's the data of about 10 comments.
Well, you left a cryptic comment that multiple people couldn’t understand. And then you refused to explain it.
Thanks for your contribution. The goal here is to enlighten each other, not expose ownership of knowledge that one has but doesn't want to share.
The gap between your 1200 Elo in 2KB and the TCEC 4K engines at ~3000 Elo is interesting. That extra 2KB buys a lot when it goes to better evaluation and move ordering. Even a simple captures-first sort in alpha-beta pruning costs only a few bytes of code but can roughly double your effective search depth.
scribbling long enough on a piece of paper is more enjoyable than prompting.
As you write: not implemented: castling, en passant, promotion, repetition, 50-move rule - those are all required to call the game being played modern chess.
I could see an argument for skipping repetition and 50-move rule for tiny engines, but you do need castling, en pessant and promotion for pretty much any serious play.
https://en.wikipedia.org/wiki/Video_Chess fit in 4k and supported fuller ruleset in 1980 did it not?
So I would ask what is the smallest fully UCI (https://www.chessprogramming.org/UCI) compliant engine available currently?
This would be a fun goal to beat - make something tiny that supports full ruleset.
PS my first chess computer in early 1980s was this: https://www.ismenio.com/chess_fidelity_cc3.html - it also supported castling, en pessant, not sure about 50 move rule.
2KB of JavaScript with castling, en passant, promotion, search and even a GUI
326 bytes of assembly, without the special rules
I don't think the author has a UCI-compliant one, but it should be easier than the GUI. There are forks of the JS one that might do it.
[0] https://nanochess.org/chess6.html