Hosted by Darius, founder of Chessengeria.eu, March 2023.
Source: LinkedIn
„(…) The chess programming field has clearly benefited from the availability of open-source engines.”
Before you is an interview with Jon Dart - the man who created Arasan - a real beast in the world of chess engines.
Arasan beats most of its competitors, and not only because of its strength, but also because it is an open project - anyone can be inspired by Jon's ideas, which he has been developing successfully for several decades!
In the interview you will learn how Jon traveled the development of his work, what ideas he had for its development and what he has planned for the future. If you are curious about what secrets Arasan hides and how it became so strong, this interview is for you!
1. Darius:
Jon, please introduce yourself to our readers.
Jon Dart:
I have lived in Silicon Valley for almost 30 years. I've had a long career in commercial software development. I've worked at a number of companies and had roles including developer, manager, and software architect. For the past 10 years I have focused on computer security. I am currently CISO at Workato, a company specializing in business integration and automation.
Arasan has been a side hobby project to my day jobs in software. In the late '80s, Windows was a new technology, and I got interested in trying to each myself C++ and Windows programming, so I had the idea of making a Windows chess program. That eventually became Arasan 1.0.
2. Darius:
Arasan is getting stronger and stronger and is capable of winning against anyone. Could you give us more details about the latest development versions of Arasan?
Jon Dart:
Arasan has become gradually stronger over time as I added more features, tuned the engine, and removed bugs.
Recently, like many engines, Arasan got a big boost in strength by adopting a NNUE evaluation. That has been worth over 100 ELO on the computer rating lists.
3. Darius:
What is the secret of Arasan's high skill level, and what long-term ideas can push Arasan's skill level to even greater heights?
Jon Dart:
The recent gain in strength is mostly due to NNUE. The current network is the result of several rounds of tuning, the most recent being a reinforcement learning pass.
I think gaining significant additional strength will probably require a different and probably larger network structure, and I have done some preliminary work on that, but it is going to take some time to develop and tune it.
4. Darius:
What are your recent major changes to Arasan over the past few years?
Is the increase in Arasan's strength the only motivator for its continuous improvement? Are there other factors ?
Jon Dart:
You can see full details in the repo logs and an overview in the CHANGES file in the repo. Some of the major changes were:
2014-2017: Auto-tuned evaluation (Texel Tuning)
2021: NNUE implementation
2022: additional NNUE tuning
Plus a lot of bug fixes and minor search/eval improvements.
I do work on improving the engine's strength, but I am not obsessed about being on top of the rating lists. I'm also just happy to see it used and enjoyed by people.
Arasan still ships with a Windows user interface, and I think the majority of users are actually downloading that package and running that version. The engine behind that isn't really optimized for strength (it is compiled for maximum portability) and anyway will be too strong for most human players. That's why the UI has a strength reduction option.
5. Darius:
What are your primary programming interests for the future of Arasan?
Is there something you are most interested in when it comes to game styles ? Do you like to create a positional / tactical / strategic engine ?
Jon Dart:
I have a lot of things in my todo list.
I don't think Arasan has a very distinctive playing style and I don't try to tune it to have a particular style. I am tuning it for overall strength. One thing I notice is that when it's playing weaker opponents, it often either is winning right out of the opening (if using its own opening book, which is pretty good), or they succumb to an attack and lose. It certainly does know how to attack and will do so if given the opportunity. But when playing other strong engines, they don't typically make opening mistakes, or if they do they can recover from them, and they don't allow spectacular attacks, in general. So those games tend to be draws.
6. Darius:
Arasan is a chess engine that we can enjoy on various operating systems, such as Linux, Mac, Windows. Is it difficult to write a chess engine in such a way that it is relatively easy to compile it for the above-mentioned systems ?
Jon Dart:
Originally Arasan had a lot of macros and hand-coded tweaks to make the code portable, but modern versions of C++ have features in the language and system libraries that provide portability, so I use those now.
The only place I do something special is setting the stack size for MacOS and Linux.
7. Darius:
You are also the author of the Arasan Test Suite, a select collection of special chess positions. Does Arasan Test Suite significantly help you develop your chess engine ? If so, what do you mainly pay attention to when using these test positions ?
Jon Dart:
I used to rely on test suites for tuning a lot, but I came to realize that they do not reliably measure program strength and in particular cannot discriminate between versions or engines that have small differences in strength. Arasan is tuned now by running fast time control games and doing a SPRT test of the new version against the old version. This became widely practiced after Houdini was said to be using that method.
Test suites are still useful to give a coarse measure of strength. I am also still interested in collecting and analyzing positions that Arasan does not solve, or does not solve quickly. Arasan is playing on several chess servers and I look at the log files from those games to find positions that deserve scrutiny.
8. Darius:
Do you have plans to introduce Arasan support for Fischer random chess (Chess960) ?
Jon Dart:
It's in the todo list, but I am not personally very interested in Chess960, and I think it would be a big job to add that support and get it thoroughly debugged, so I haven't done it so far.
9. Darius:
When can we expect Arasan 24 ?
Jon Dart:
I am going to release 23.5 soon with some minor fixes and updates. The next major release is probably going to be when I have a new network implemented, but I don't have a projected timeframe for that. I also have other things I would like to clean up: for example, most of the HCE is not used anymore, so could be removed.
10. Darius:
What is your opinion on creating a contemporary chess engine using only Hand Craft Eval (HCE). Do you see any advantages and the future of such an approach here, considering the possibility of using a neural network (NNUE) ?
Jon Dart:
I think people just beginning with chess programming can still use that for a start and add NNUE later.
Both Texel-type tuning and NNUE developments have shown that the eval is very important. I had a real hand-coded evaluation for many years, and it was pretty good I thought, but then auto-tuning it gave a significant boost in strength, and NNUE an even greater boost. I think some form of NNUE is necessary for top-level performance now.
11. Darius:
The number of commercial chess engines has significantly decreased in recent years. Instead, there are an increasing number of free open source chess engines. Is this a good or bad thing for computer chess in general, and for users in particular ?
Jon Dart:
Many of the commercially successful products like Fritz have nice UIs and features that go beyond just the engine. So there is a place for those. The market for super-strong engines by themselves is fairly small, I think, and only the very top engines attract buyers.
The chess programming field has clearly benefited from the availability of open-source engines. When I started out in chess programming, I read what books and articles there were on the subject. But source code is very helpful to see how ideas are implemented. There were very few open source engines in the early days. The main one was Gnuchess (the original one). But I found Gnuchess not a very good model to follow. The code was dense and had few comments and it had some quirky features. Don't get me wrong: it was a good program, it was just having the source did not help me much. A little later Crafty came along and it was more clearly understandable to me. I followed Crafty development quite closely and was always trying to get better than it, but that was a hard task for a long time, because Bob Hyatt kept improving it.
Over the years a lot more open source engines appeared, and I found many of them interesting. I remember looking at Toga and being amazed at how aggressive its pruning was. I thought: how could this possibly work? But it was a strong engine for its time, and that level of pruning is now common.
I will mention though that unlike most engines, mine is MIT-licensed, not GPL. So that allows derivatives of the engine to be distributed with attribution, but without including source. I am aware that there are some commercial projects using the code.
12. Darius:
Where do you see the key to the strength of engines like Stockfish, Dragon or Berserk ? The aforementioned trio of chess engines are currently (2023) mostly thundering their competitors, clearly ahead of them in strength measured in Elo points.
Jon Dart:
Stockfish has benefited a lot from its large community of contributors. They try a very large number of changes and only a few of the best make it into the main branch. Stockfish also benefits from its distributed testing network. I have several large servers to test on, but Stockfish has a much bigger pool and can do test runs of 100,000 games or more. They have made a lot of progress by putting in a lot of small incremental changes.
Dragon I don't have much familiarity with. Berserk as far as I can tell doesn't have a lot of unique features, except its NNUE is different from other engines.
13. Darius:
Do you have any favorite chess software with which you test your chess engine ?
Jon Dart:
I used to run offline matches against other engines for testing, but now most of my testing is self-play testing (old vs. new version of Arasan). I use cutechess-cli and some custom scripts for running matches across a pool of machines and monitoring the results.
As mentioned, Arasan plays on the chess servers (FICS, ICC and lichess, currently) and I find those games interesting and sometimes helpful for finding bugs and ideas for improvement.
14. Darius:
Arasan is a well-documented chess engine, and its website has been, and continues to be, an example of extreme meticulousness and attention to detail for many years. We can say the same in the context of Arasan's development. Please tell us what is behind such exceptional consistency and determination in achieving goals and successfully enhancing the capabilities of the Arasan engine ?
Jon Dart:
Well, personally I just find it challenging and interesting to work on, and that challenge and interest hasn't gone away over time.
Darius:
Jon, on behalf of Chessengeria's readers and myself, I thank you for participating in the interview and sharing your experience of developing one of the best chess engines in the world.
I wish you continued success in your work on Arasan and may your passion accompany you for the next several decades!
NOTE:
Further publication and reproduction of this interview is permitted only with the prior permission of Darius, founder of chessengeria.eu
The condition is that the content is preserved in its entirety and that a link to chessengeria.eu is included.
In the Files area you will find free chess engines for download.