Practice Chess Openings with Anki Flashcards

Syzygy endgame tablebases are a popular choice when the endgame capabilities of a chess engine should be improved with tablebase lookups. But this is not a low-hanging fruit. It is in fact quite easy to destroy performance completely, if you are not careful.

Characteristics of Syzygy Tables

Most people would expect and endgame tablebase to answer to questions for a given position: What is the outcome of this position? And what is the best move in this position? Syzygy tables can give an answer to first question. However, they do not tell you what the best move is. But in the following, we will see how you can find a good move with them.

Syzygy tables have been generated for all chess positions that with 3 to 7 pieces remaining on the board. For all positions? Not quite. First of all, the tables have been created under the assumption that both sides have lost their castling rights. The reason for that was to save space, and such positions are rare anyhow with 7 pieces or less.

It was also assumed that the last move leading to the position was a pawn push or a capture so that the halfmove counter for the 50-move-rule is zero. That does not mean that you cannot use the tables, when the counter is non-zero, but you have to add some additional calculations to interpret the result of the probe correctly.

Take for example this position:

. . . . . . . k / . . . . . . . . / . . . . . . . . / . . . B N . . . / . . . K . . . . / . . . . . . . . / . . . . . . . . / . . . . . . . .

If you are playing chess, you know that this is a theoretical win for white but the mate is hard to find. A table query shows you that this is a mate in 48 moves, see https://syzygy-tables.info/?fen=7k/8/8/3BN3/3K4/8/8/8_b_-_-_0_1.

However, if what has already fruitlessly moved around their pieces for more than two moves, this is a draw, because black can claim a draw according to the 50-move-rule, once 50 moves have been made without a pawn push or capture.

Another aspect to keep in mind is draw by repetition. The rules of chess state that the side to move can claim a draw, if they can make a move that produces the same position for the third time. Example:

k . . . . . . . / . . . . . . . . / . K . . . . . . / . R . . . . . . / . . . . . . . . / . . . . . . . . / . . . . . . . . / . . . . . . . .

You can easily see that this is a mate in 2 for white, for example 1. Rh5 Kb8 2. Rh8#, see https://syzygy-tables.info/?fen=k7/8/1K6/1R6/8/8/8/8_w_-_-_0_1

However, chess is not stateless. There is a game history. Imagine that in the current position, these moves are played: 1. Rh5 Kb8 2. Rb5 Ka8 3. Rh5 Kb8 4. Rb5 Ka8, in other words white moves the rook back and forth, and black moves the king back and forth. If white where now to play 5. Rh5, black could immediately claim a draw, because it has 5...Kb8 which repeats the position for the third time. It does not help white that they can play 6. Rh8#. This is another thing that you have to bear in mind.

After this gentle introduction, let's look into the technical details.

Win/Draw/Loss - WDL

There are two kinds of Syzygy tables. We first look at the WDL tables. A probe of a WDL table gives you the outcome of a game for any position with one of these values:

• +2: a win
• +1: a "cursed" win
• 0: a draw
• -1: a "blessed" loss
• -2: a loss

A cursed win is a position with a forced mate sequence that is longer than 50 moves. An example is this position:

. . . . . . . . . . . . . . . . . . . k . . . . . . . . . . . . . . . K . . . p . . . . . . . N . . . . . . . . . . . . . . N .

See https://syzygy-tables.info/?fen=8/8/3k4/8/3K3p/7N/8/6N1_w_-_-_0_1.

White can force a mate but it takes 85 moves. That means, under the 50 move rule, this is a draw, not a win. Such cases are not purely theoretical, they really occur. The example position is one from a game of Magnus Carlsen against Maxime Vachier Lagrave.

By the way, the best move in this position is Nf3, and the resulting position is a "blessed loss" for black. It can escape the defeat by claiming a draw according to the 50-move-rule after 49 more moves.

Distance to Zero - DTZ

The other kind of tables are the DTZ tables that give the "distance to zero" for a position. Zero refers to the halfmove counter and distance to zero means the number of halfmoves following until a pawn is pushed, a piece is captured or one side is mated.

Because the result of the probe is a signed integer, this is a superset of the information that the WDL table gives:

• = +100: a cursed win

• = +1: a win

• 0: a draw
• <= -1: a loss
• <= -100: a blessed loss

So why not always probe the DTZ tables?

DTZ tables are a lot bigger and it takes a lot longer to probe them. In practice, the DTZ tables are only used for ordering and pruning moves at the root, whereas WDL tables are used in the normal (alphabeta) search.

Hardware Aspects

Since WDL tables are probed a lot of times during the search, they must be stored on a fast medium, either a RAM disk or on a fast SSD. Putting them on a conventional spinning disk will not work. It is simply too slow.

The DTZ tables can be stored anywhere without having too much of an impact on the performance of the engine. One problem, however, are disks going to sleep. An external harddisk connected via USB often takes several seconds to wake up if has not been accessed for a certain time. Trying to prevent them from going to sleep is difficult in practice. You will probably be better off setting an alarm, and if the probe takes too long, just wait for the next move, when the disk will hopefully still be ready.

Incomplete Tablebases

It is theoretically possible that there is a tablebase file for 7 pieces but one or more 6-piece tables are missing. It is probably best to treat that as a configuration error, abort tablebase probing for the current search and proceed with a regular search. Handling these exceptions is too expensive, and such setups do not make any sense.

On the other hand, it is common and possible, to have only WDL tables. These tables are already sufficient for move ordering or pruning, although the results will be less accurate.

Mating Strategies

When you only have the distance to zero, how can you find the best move then? For that, let's assume that the position is a tablebase win. We will later see that the other two cases are trivial to solve.

Maximising WDL and Minimising DTZ

Let's recap, what a "zeroing" move is. It is a move that has at least one of the following properties:

• it pushes a pawn
• it captures a piece
• it mates

There is only a finite number of pawn pushes and captures possible. Therefore, minimising the DTZ will always inevitably lead to a mate. At one point there are no more pawns to push or another capture would lead to a draw by insufficient material.

Therefore, one seemingly primitive strategy is to simply minimise the DTZ. This will often lead to funny moves like in this position:

. . . . . . . . . . . . . . . . R . . . . . . . R . . . . . . . R . . . . . . . R . . . . . . . R . . . . . . . K . . . . . . k

When white minimises DTZ, the next few moves could be 1. Rh2+ Kxh2 2. Rh3+ Kxh3 3. Rh4+ Kxh4 White would sacrifice four of their five rooks because the captures will set the halfmove counter to 0. Once, there is just one rook left, it would start making "intelligent" moves and still win. You will probably also understand that accepting the sacrifices is actually wrong for black.

Root Move Ordering Plus Search

The common approach is, however, to use the DTZ information only for ordering the root moves. It is well known that an alphabeta search is most efficient, when the best moves are searched first, and the moves with a low DTZ have a higher potential to be the best move. It also favours in a natural way pawn pushes and captures because they are zeroing moves.

From there on, the engine just does a normal search, and tries to find the optimal move.

Tablebase Pruning

Tablebases serve as an oracle. They give knowledge about a certain position without doing a search. This can be used for pruning moves that cannot be the optimal move.

For example, when the WDL score of a position is +2, then there must be at least one move that results in a WDL score of -2. The sign is reversed, because the socrer for that position is from the other side's perspective.

There may be moves with a WDL score of -1, 0, or even worse +1, or +2, but these moves cannot be the best move. It is therefore a waste of time to search them at all.

For MultiPV engines, this may cause a little bit of a problem because the engine is supposed to display as many principal variations as requested. If you think that this is a problem, just pick as many inferior moves as needed, assign a tablebase win as an upper bound to them, and avoid searching them.

The same technique can be used inside of the search. Whether this makes sense is subject to experimentation. It probably makes sense if the start position of the search is a tablebase hit, because you know that the real PV will always have alternating WDL values after each move.

Root Move Ordering

Most engines seem to have just one function for root move ordering (and maybe a second one for root move ordering without DTZ tables). In my opinion to have three distinct routines for the root move ordering and pruning, depending on whether the root WDL probe gave a win, a draw, or a loss. Having distinct routines for the three cases, will make the code easier write and also easier to read, and it will save a lot of branching.

Losing Positions

Let's start with losing positions, positions with a WDL score of -2 or -1.

Remember that the ultimate goal of the engine is to find the best move for any given position. If the tablebase oracle has already reveiled that the opponent can force a win, what should the strategy of the losing side be? It should try to avoid the mate for as long as possible, because you want to give the opponent as many chances as possible to blunder or lose on time.

To achieve this goal is ridiculously easy with DTZ tables, when you bring to mind a trivial formula of chess:

$\mathrm{DTM}\ge \mathrm{DTZ}$

This is easy to understand from the winning side. The mating move, the end of the mate sequence is always a zeroing move. That means that the DTZ can never be greater than the DTM. The same formula is true for the losing side, because DTMs and DTZs are conventionally negated for them.

Should a WDL probe of the resulting position after every move be done? Normally not, because the result is already known. It is +2. The oracle has found out that the current position is a loss. And that means that there is no move that can turn that loss into a draw or win. There can also be no move that can turn the position into a blessed loss (which is a draw under the 50-move rule).

However, if you want to handle losses and blessed losses with the same code, you have to use a little formula. In pseudocode:

DTZ2WDL(dtz) {
    if (dtz >= 100) {
        return +1
    } elsif (dtz > 0) {
        return +2
    } elsif (dtz <= -100) {
        return -1
    } elsif (dtz < 0) {
        return -2
    } else {
        return 0
    }
}

And what if there are no DTZ tables? Then it is impossible to find the best move with tablebases and the normal move ordering should be done instead.

What about blessed losses? The root probe gave a score of -1. Then simply prune all moves that result in a WDL score of +2 because these moves would blunder a blessed loss into a regular win for the opponent.

The resulting pseudo-code will show that it is not necessary to distinguish between regular losses and blessed losses. The exact same strategy will apply:

orderRootMovesLosing(rootWDL) {
    for move in legalMoves {
        position.doMove(move)
        dtz = probeDTZ(position)


        position.undoMove(move)


    }
}