Candidates Tournament

 The winner will face Magnus Carlsen in the World Championship

Norway Chess 2022 Update

 Updated forecast after Tari replaced Rapport

Norway Chess 2022

 Anand returns to classical chess! Though many players are skipping the tournament in order to focus on the Candidates, the field is still very strong.

Are 2700s overrated? Insights from the new model

In my last post, I described the new model. Now we will look at more results, though this time I will try to stay away from all the technical jargon.

If you perform better than your expected score, then your rating goes up. But is the expected score formula accurate? The model can tell us. Here is the graph for a 2650 player facing opponents from 2600 to 2900. When a 2650 plays another 2650, the expected score is 0.5. Not surprising - when you play someone with the same rating, you have equal chances. The model and Elo's formula agree on that. But then the two lines diverge. On the far left, we have a 2650 playing a 2600. Elo's formula ("theory") says that the 2650's expected score is about 0.57. But in the data, the 2650 scores slightly worse - roughly 0.55. This means that in real life, the 2650 will lose rating points if playing weaker players. On the other side of the graph, the pattern reverses. When facing stronger opponents, 2650s perform better than their rating and gain points.

The next graph is for a 2700. It is a similar story. 2700s underperform when facing weaker opponents. However, they do better than expected against stronger players.

It's also true for 2750s:

What does this mean for the forecasts? The new model is based on the data, so it will give more weight to the underdogs. The old model was based on Elo's formula; it was overestimating the favorites. We can see this when we revisit the Carlsen-Nepo match. Carlsen was the higher rated player, so the original model gave him higher chances.

Old model

Carlsen wins the match: 82.86%

Nepo wins the match: 9.115%

Drawn match: 8.025%

New model

Carlsen wins the match: 75.535%

Nepo wins the match: 13.56%

Drawn match: 10.905%

A word of caution about interpreting the results: the model was based on games with at least one 2700 player. Will a 2000-rated amateur gain points if they play up a section? I don't know. The model was designed for elite tournaments, not amateurs. 

I don't have a clear cut answer for the question in the title. Are 2700s overrated? Yes - but only when playing weaker opponents. When playing stronger opponents, they are underrated. Should FIDE fix this problem by adjusting the expected score formula? It might be more complicated than that. There is going to be a feedback effect on the ratings. There are many rating systems that are superior to Elo; it would be better to switch to one of them instead.

New Methodology

 Before the Carlsen-Nepo match, I had noticed that my model was underestimating the draw rate. I improved it before the World Championship, but there was more work to do. The original model was based on a large database. Most of the games involved strong amateurs, such as 2300s and 2400s. Elite tournaments were a tiny minority, so the model was not focused on them. However, my forecasts are always for top tournaments. First, I needed a better sample. I only kept games where both players were 2500+. Then I estimated a model based on games where at least one player was 2700+ during the period 2010-2021.

The rest of the details are more technical. Each chess game has 3 possible outcomes; this suggests that an ordered logit model is appropriate. However, several sources said that the proportional odds assumption is often violated in the data. Instead, I used a generalized order logit. The explanatory variables were:

-the rating gap (i.e. White's rating - Black's rating)

-the average rating of the two players

-the year in which the game was played

-"elite": this variable equals 1 if one of the players is 2750+ and other other isn't. Otherwise it equals 0. This variable could matter if 2750's are overrated. Normally if a player is overrated, then their rating should adjust. When their rating is above their true strength, they won't be able to perform well enough to maintain it. However, top players mostly compete in round robins against each other. They don't mix with the rest of the pool. If overrated players face each other, then the ratings don't adjust.

-interaction terms (gap x avg, gap x year, avg x year, elite x gap, elite x avg, elite x year, gap^2, avg^2, year^2)

The terms involving Elite were statistically significant (p-value = 0.0004). Here are the results (N = 25534 games):

Interpreting ordered logit coefficients is notoriously difficult. Based on this table, I can't say if Elite players are overrated or underrated, but the Elite variable is clearly significant. 

Next, I checked if the model fixes the original problem: does it underestimate the draw rate? If both players are 2750+, the model's draw rate is only 0.19% lower. This difference is statistically insignificant (p-value = 0.832). Though the model is targeted for 2700s, sometimes an organizer will invite a local 2600 to an elite round robin. Does the model still work for them? I tested it in this sample: one player is 2700+, the other isn't. The model's draw rate is 0.14% too high, which is statistically insignificant (p-value = 0.709).

The new model estimates the win, loss, and draw probabilities. The traditional model only estimated the draw rate; the win and loss probabilities were then extrapolated from Elo's expected score formula. This allows me to test the validity of the expected score formula. I will discuss this in a different post, since most readers probably gave up somewhere around the second paragraph.

2022 Tata Steel Masters

 I changed the model before the World Championship, but I'm still not entirely happy with it. In elite tournaments, the draw rate seems to have risen in the last decade or so. My model does account for the year in which the game was played, but its predicted draw rate remains too low for recent tournaments. No forecast - I'm still making changes.

Stockfish + Bongcloud vs. 2100

[Event "Stockbong Challenge"] [Site "Binghamton"] [Date "2022.01.01"] [Round "1"] [White "Stockfish 14.1"] [Black "Wilson, Matthew"] [Result "1-0"] [BlackElo "2127"] [SetUp "1"] [FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"] [PlyCount "109"] [EventDate "2022.??.??"] 1. e4 c5 2. Ke2 {The Stockbong challenge: I made Stockfish play the Bongcloud. It's move 2 and I already have a big advantage, so maybe I can hope to survive?!} d5 3. exd5 Qxd5 4. Ke1 e5 5. Qf3 Nf6 $2 {I took the wrong approach to this game. Black needs to keep pieces on the board since White can't castle. But I was happy to trade down towards a draw. I did look at 5...Qd6. However, I didn't like 6.Bc4 followed by Nc3 and Ne4 - White would get trades anyways and his pieces would be centralized.} (5... Qe6 {(SF) is stronger.} 6. b3 Nc6 7. Bc4 Qe7 8. Nc3 Nd4 {is great for Black}) 6. Qxd5 Nxd5 7. Nc3 Be6 8. Nxd5 ({ I was planning to meet} 8. Bc4 {with} Nf4) 8... Bxd5 9. Ne2 Nc6 10. Nc3 Be6 11. Bb5 Rc8 {I thought it was important to avoid doubled c-pawns. I did consider 11...Kd7, but I felt that the text was better. The rook commonly goes to c8 in these Maroczy bind structures. If instead 11...Kd7, I will probably have to spend a tempo on ...Kc7 sometime in the future} 12. d3 Be7 13. b3 O-O 14. Bc4 Nd4 15. Kd2 Bg5+ $6 {Giving away my advantage. Stockfish prefers 15...f5.} 16. Kd1 Bxc1 ({At first, I thought I was winning material with} 16... Bg4+ 17. f3 Nxf3 {, but then I spotted} (17... Bxc1 18. Kxc1 { and now there is no fork on f3}) 18. h3 $1 ({not} 18. gxf3 $2 Bxf3+ 19. Ne2 ( 19. Ke1 Bxc1 $1 20. Rf1 Bb2 $1) 19... Bxc1 $1 20. Rf1 Bxe2+) 18... Bh5 19. Bxg5 Nxg5+ 20. g4 Bg6 (20... Nxh3 {(SF) is better, but after} 21. gxh5 Nf2+ 22. Kd2 Nxh1 23. Rxh1 {, the minor pieces outweigh the rook}) 21. h4 { followed by 22.h5, trapping the bishop}) 17. Kxc1 Rfd8 18. a4 ({ During the game, I wasn't sure about} 18. Re1 f6 19. f4 Bxc4 20. bxc4 exf4 { , but Stockfish shows that Black is at least equal here:} 21. Nd5 (21. Re7 $2 Re8 $1 22. Rxb7 Re1+ 23. Kb2 Rxa1 24. Kxa1 Nxc2+ {followed by ...Nc2-e3xg2}) 21... Re8 $1 {Now 22.Ne7+ doesn't work, so White has to play 22.Kd2 and try to recover the pawn}) 18... f6 19. Nb5 Bxc4 ({My original intention was} 19... a6 20. Nxd4 Bxc4 {but then I saw} 21. Nf5 $1 {winning material}) 20. bxc4 Nxb5 { A concession} (20... Kf7 {is best (SF). The tactical justification is} 21. Nxa7 Ra8 22. Nb5 Nxb5 23. cxb5 Ra5 $1 {and Black will recover the pawn}) (20... Nc6 {is simple and White doesn't have much}) 21. axb5 Ra8 22. Kd2 Rdb8 (22... a5 $2 23. bxa6 Rxa6 24. Rxa6 bxa6 25. Rb1 {and White dominates the open file. The text aims to open the b-file under more favorable conditions}) 23. h4 ({ I was expecting} 23. Rhb1 {in order to prevent ...a5. But Stockfish correctly judges that ...a5 weakens my position}) 23... a5 $6 {This gets me in trouble} ( 23... b6 24. Ra6 Rb7 25. Rha1 { and my rooks are very passive. However, White can't break through:} h5 26. Ke3 Kf7 27. Ke4 Ke6 28. c3 (28. f4 exf4 29. d4 Kd6) 28... g5 {. Even if the White king could somehow get to d5, I could reorganize my defense with ...Rd8+ ... Rdd7}) 24. bxa6 Rxa6 25. Rhb1 (25. Rxa6 bxa6 26. Ra1 Rb6 27. Ra5 Rc6 { and Black should be able to hang on}) 25... Rxa1 26. Rxa1 Kf7 27. Ra5 Rc8 $2 { The decisive mistake, which I played quickly} ({I rejected} 27... b6 28. Ra7+ { since I didn't want to surrender the 7th rank. But Stocky shows that Black can hold:} Kg6 29. Rc7 (29. Ke3 Rd8 {followed by ...Rd6}) 29... h5 30. Ke3 Ra8 31. Rb7 Ra6 {Black's pieces look awkward, but White can't move forward without allowing counterplay.} 32. Ke4 (32. g3 Kh6 33. Ke4 Ra2 34. Rxb6 Rxc2 35. Ke3 e4 $1 36. Kxe4 Rxf2 37. d4 cxd4 38. c5 Kg6 39. Kxd4 Kf5 40. Rb3 Ke6 {triple zeros} ) 32... Ra2 33. Rxb6 Rxc2 34. Ke3 Kf5 35. Rc6 g5 36. hxg5 fxg5 37. Rxc5 Kf6 { and White can't make progress}) 28. Rb5 Rc7 29. Ke3 Ke6 30. Ke4 {I was expectin g 30.g4, but as we will see in the next note, White is not afraid of ...f5+} Rd7 {Sacrificing a pawn for activity} (30... f5+ 31. Ke3 Kf7 32. f4 exf4+ 33. Kxf4 {and the White king will invade on the dark squares}) 31. Rxc5 Rd4+ 32. Ke3 Rxh4 33. Rb5 Rg4 34. g3 h5 35. Rxb7 g5 {Trying to create a passed h-pawn, but Stockfish quickly turns it into a weakness} 36. Rh7 h4 37. gxh4 gxh4 38. c3 Kd6 {I spent a lot of time on this move.} (38... Kf5 {was the main alternative. The idea is to kick out the White rook so I can push the h-pawn} 39. f3 $1 Rf4 40. Rh8 $1 Kg6 41. c5 Kg7 42. c6 $1 {and the pawn promotes}) 39. f3 $1 Rf4 { Now my "active" rook is nearly trapped. I hoped that it would keep White's king cut off, but itisn'tenough} ({I looked at} 39... Rg2 40. Rxh4 Rc2 { , but I didn't like my position after} 41. f4 {. Either I let White have connected passers or my e5-pawn is weak. Stocky's main line continues} Rxc3 42. fxe5+ fxe5 43. Rh6+ Kc5 44. Re6 {and White wins}) 40. d4 { Now the passers just march down the board} Ke6 41. c5 Kd5 42. Rh8 Kc6 43. Rc8+ Kd7 44. Rf8 Rf5 ({I thought} 44... Ke7 {lost to} 45. dxe5 { , but Stockfish finds} Rf5 $1 {. Instead 45.c6 wins}) 45. d5 Rf4 46. c6+ Kd6 47. Rd8+ Kc7 48. Rd7+ Kc8 49. Rh7 Rc4 50. Kd3 Rc5 51. c4 f5 52. Kc3 e4 53. Kb4 exf3 54. Rh8+ Kc7 55. Kxc5 1-0