V36 looks weaker than V35.3

[arturoiochoam]

Describe the issue

It may be premature to draw definitive conclusions at this stage, as 100 games may not provide a sufficiently large sample size. However, it appears that Version 36 has shown a decline in performance compared to Version 35.3. In my initial testing at a 2-minute 3-second increment time control, Version 36 has achieved less favorable results against a Stockfish-clone engine known as Vanilla 15.

ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Vanilla 15 6t1024h : 100 : 2+ : 97= : 1- : 50.5%

ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Vanilla 15 6t1024h : 100 : 4+ : 81= : 15- : 44.5%

An example of a lost game against Stockfish 11:

Expected behavior

V36.0 should be at least as strong Stockfish 17

Steps to reproduce

See how the endage of this game.

[Event "Vafra Files"]
[Site "i7-13700HX 2.10 GHz 16C-24T, 32.0 GB"]
[Date "2024.09.23"]
[Round "9"]
[White "Stockfish 11 6t1024h"]
[Black "ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h"]
[Result "1-0"]
[ECO "A06"]
[Opening "Reti Opening"]
[Time "15:44:32"]
[Variation "Old Indian Attack, 1.Nf3 d5 2.d3 Nf6"]
[TimeControl "120+3"]
[Termination "normal"]
[PlyCount "271"]

1. d3 d5 2. Nf3 {+0.29/26 7} Nf6 3. Bf4 {-0.01/27 24} c5 4. e3 {-0.30/26 5}
   Qb6 5. Qc1 {-0.12/24 2} Nh5 6. Be5 {0.00/26 8} Nc6 7. d4 {-0.06/23 2} Nf6
2. a4 {+0.10/25 8} e6 9. c3 {+0.04/27 2} Nxe5 10. Nxe5 {0.00/26 2} Bd6 11.
   Bb5+ {0.00/26 2} Kf8 {+0.23/27 3} 12. Be2 {+0.08/26 3} g6 {+0.27/30 6} 13.
   f4 {+0.10/27 11} Kg7 {+0.24/28 6} 14. Nd2 {+0.14/26 6} Qc7 {+0.21/33 9} 15.
   O-O {+0.16/23 2} a5 {+0.23/34 24} 16. Qe1 {0.00/30 11} Bd7 {+0.23/35 3} 17.
   Bd3 {0.00/24 13} Be8 {+0.29/30 5} 18. Qf2 {+0.39/24 4} Rf8 {+0.31/37 50}
3. g4 {0.00/30 8} Ng8 {+0.35/43 3} 20. g5 {+0.61/23 1} h5 {+0.40/29 3} 21.
   Rfc1 {+0.67/25 8} Ne7 {+0.40/29 3} 22. b3 {+0.65/26 4} Rb8 {+0.43/30 4} 23.
   Ndf3 {+0.60/26 3} cxd4 {+0.50/29 3} 24. cxd4 {+0.49/26 5} Bc6 {+0.40/45 21}
4. Qe1 {0.00/31 22} Qb6 {+0.39/38 8} 26. Rab1 {0.00/31 5} Rfc8 {+0.38/40
   8} 27. Kf2 {0.00/32 1} Be8 {+0.41/40 2} 28. Qd1 {0.00/31 2} Kg8 {+0.36/27
   7} 29. Ra1 {0.00/33 4} Qb4 {+0.35/32 9} 30. Be2 {0.00/34 2} Qb6 {+0.35/32
   4} 31. Bd3 {0.00/34 2} Kf8 {+0.35/24 2} 32. Ng1 {0.00/31 2} Rxc1 {+0.47/25
   2} 33. Rxc1 {0.00/34 3} Qb4 {+0.51/10 2} 34. Ne2 {0.00/31 5} b5 {+0.52/10
   1} 35. Bc2 {0.00/32 2} Bxe5 {+0.52/29 3} 36. fxe5 {0.00/33 2} Rc8 {+0.56/25
   3} 37. Rb1 {0.00/31 3} Kg7 {+0.58/29 5} 38. Ra1 {0.00/36 8} Nc6 {+0.64/10
   2} 39. axb5 {0.00/33 1} Ne7 {+0.65/10 2} 40. Ra4 {0.00/31 2} Qxb5 {+0.65/10
   2} 41. Qd2 {0.00/39 2} Qb6 {+0.34/36 5} 42. Ra2 {0.00/41 3} Bb5 {+0.31/33
   6} 43. Nf4 {0.00/37 2} Nc6 {+0.30/12 2} 44. Bd1 {0.00/37 2} Nb4 {+0.41/23
   2} 45. Ra1 {0.00/39 2} Rc6 {+0.37/20 2} 46. Rc1 {0.00/34 4} Rxc1 {+0.40/24
   2} 47. Qxc1 {0.00/32 12} h4 {+0.41/12 2} 48. Kf3 {0.00/33 1} Nd3 {+0.40/19
   2} 49. Qd2 {0.00/42 2} Nxf4 {+0.40/12 2} 50. Kxf4 {0.00/40 2} Qa6 {+0.52/9
   2} 51. Kg4 {0.00/44 1} Bd3 {+0.19/41 5} 52. Kxh4 {0.00/42 2} Bf5 {+0.23/12
   2} 53. Qe1 {0.00/43 2} Qd3 {+0.15/38 6} 54. Kg3 {0.00/45 2} Qe4 {+0.10/33
   2} 55. h3 {0.00/44 4} Kh7 {+0.20/31 2} 56. Kh2 {0.00/37 2} Kg7 {+0.15/41 3}
5. Kg3 {0.00/39 2} Kh7 58. Qf2 {0.00/38 2} Kg8 {+0.12/31 2} 59. Qe1
   {0.00/42 10} Qd3 {+0.10/34 2} 60. Kh2 {0.00/43 1} Be4 {+0.06/42 3} 61. Kg3
   {0.00/42 2} Bf5 {+0.08/16 2} 62. Bf3 {0.00/35 1} Kg7 {+0.05/32 2} 63. Bd1
   {0.00/44 5} Be4 {+0.04/41 3} 64. Kf2 {0.00/41 1} Bf5 {+0.03/30 2} 65. Kg3
   {0.00/43 2} Be4 66. Kf2 {0.00/43 1} Qb1 {+0.03/33 2} 67. Qe2 {0.00/42 3}
   Bf5 {+0.12/44 2} 68. Kg3 {0.00/43 4} Qc1 {+0.31/25 2} 69. h4 {0.00/41 4}
   Qc3 {+0.12/54 7} 70. Kf2 {0.00/41 4} Bd3 {+0.08/23 2} 71. Qe1 {0.00/38 2}
   Qb2+ {+0.06/31 2} 72. Kg3 {0.00/40 2} Bf5 {+0.06/38 4} 73. Qe2 {0.00/39 1}
   Qc3 {+0.15/24 2} 74. Kf2 {0.00/38 4} Bd3 75. Qe1 {0.00/44 1} Qb2+ 76. Kg3
   {0.00/37 1} Qc1 {+0.02/40 3} 77. Kf2 {0.00/40 2} Bf5 {+0.04/34 2} 78. Qe2
   {0.00/42 4} Qc3 {0.00/76 3} 79. Qe1 {0.00/41 2} Qb2+ {0.00/129 2} 80. Qe2
   {0.00/42 5} Qc3 {0.00/133 3} 81. Qe1 {0.00/36 1} Qd3 {0.00/103 3} 82. Kg3
   {0.00/40 4} Qb1 {0.00/99 3} 83. Qe2 {0.00/38 2} Bd3 {0.00/145 3} 84. Qd2
   {0.00/45 1} Bf5 {0.00/173 3} 85. Kh2 {0.00/41 1} Be4 {0.00/161 3} 86. Kh3
   {0.00/48 2} Bf5+ {0.00/155 3} 87. Kg3 {0.00/45 1} Kg8 {0.00/128 3} 88. Qe2
   {0.00/47 4} Qc1 {0.00/130 3} 89. Kf2 {0.00/47 6} Kg7 {0.00/143 3} 90. Kg3
   {0.00/43 1} Kg8 {0.00/76 2} 91. Kf2 {0.00/44 2} Be4 {0.00/103 4} 92. Qe1
   {0.00/35 1} Bd3 {0.00/111 2} 93. Bf3 {0.00/37 1} Qb2+ {0.00/136 2} 94. Kg3
   {0.00/38 2} Qa3 {0.00/90 2} 95. h5 {0.00/26 3} gxh5 {0.00/46 2} 96. Bxh5
   {0.00/27 2} Qb4 {0.00/75 2} 97. Qc1 {0.00/26 4} Bf5 {0.00/33 2} 98. Kf4
   {0.00/25 6} Qb7 {0.00/116 3} 99. Qd2 {0.00/27 2} Qb6 {0.00/80 14} 100. Qg2
   {0.00/27 3} Qxb3 {+0.93/1 1} 101. g6 {+2.60/19 1} fxg6 {-1.61/17 2} 102.
   Bxg6 {+2.72/22 3} Kf8 {-1.49/12 2} 103. Bxf5 {+3.30/24 2} exf5 {-1.61/23 2}
6. e6 {+3.38/26 2} a4 {-0.84/10 2} 105. Qg6 {+3.60/29 3} Qb8+ {-2.64/27
   3} 106. Kf3 {+3.85/28 1} Qe8 {-2.24/17 2} 107. Qxf5+ {+4.26/29 3} Ke7
   {-2.82/35 5} 108. Qxd5 {+4.34/31 1} a3 {-2.82/25 2} 109. Qc5+ {+5.30/30 6}
   Kxe6 {-2.84/19 2} 110. Qxa3 {+5.90/31 4} Qh5+ 111. Kg3 {+6.07/27 1} Qg5+
7. Kf2 {+6.19/30 1} Qh4+ 113. Ke2 {+6.73/28 3} Qg4+ 114. Kd2 {+7.51/29 5}
   Qg2+ 115. Kc3 {+7.90/25 1} Qe4 116. Qa6+ {+8.68/25 2} Ke7 117. Qa7+
   {+9.55/25 1} Ke6 118. Qb6+ {+11.15/27 5} Kf7 119. Qc7+ {+11.43/25 1} Ke6
8. Qc8+ {+11.72/25 1} Ke7 121. Qc5+ {+12.61/26 2} Ke6 122. Kc4 {+13.81/26
   2} Qc2+ 123. Kb5 {+147.92/28 1} Qb3+ 124. Kc6 {+147.92/31 3} Qa4+ 125. Kc7
   {+148.86/29 7} Qd7+ 126. Kb6 {+148.95/30 1} Qd8+ 127. Kb7 {+148.96/28 1}
   Qd7+ 128. Qc7 {+M54/42 6} Qa4 129. Qc6+ {+M18/42 1} Kf5 130. Qxa4 {+M8/61
   1} Ke4 131. d5+ {+M7/84 1} Kd3 132. d6 {+M6/78 1} Kd2 133. d7 {+M5/141 2}
   Kc1 134. d8=Q {+M3/166 2} Kb1 135. Qd3+ {+M2/150 2} Kc1 136. Qa1# {+M1/165
   2} 1-0

Anything else?

No response

Operating system

Windows

ShashChess version

Version 36

[arturoiochoam]

I also notice that using the Lichess causes issues because it contains opening blunders. I have disabled all the Lichess option and left only Chessdb.

[arturoiochoam]

Also reported in Talkchess that Shashchess v36 and Brainlearn v29 has multimples looses on time: https://www.talkchess.com/viewtopic.php?p=969423#p969423

[arturoiochoam]

Hello, after increasing the overhead time to 1000s, I still experience looses on time. Finally, I disable any consultation to Lichess and only preseving enabled the options to ChessDB and the looses by time are gone.

[arturoiochoam]

Next, the recent result with v36:

ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Alexander 1.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h :  100  :  37+  :  63=  :   0-  :   68.5%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Beast 15 6t1024h :  100  :   0+  :  99=  :   1-  :   49.5%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Brainlearn 29 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h :   48  :   1+  :  47=  :   0-  :   51.04%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Chess-System-Tal 2.00.v21-E1019 6t1024h :  100  :  14+  :  86=  :   0-  :   57.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Clover 6.2 6t1024h :  100  :  21+  :  79=  :   0-  :   60.5%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Clover 7.1 6t1024h :  100  :  15+  :  84=  :   1-  :   57.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Coiled 1.2 6t1024h :  100  :  44+  :  56=  :   0-  :   72.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Lc0.cuda 0.31.1 611246.384 6t1024h :  100  :  18+  :  82=  :   0-  :   59.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Lc0.cuda 0.31.1 715893.256 6t1024h :  100  :  47+  :  53=  :   0-  :   73.5%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Lc0.onnx-dml 0.31.1 715893.256 6t1024h :  100  :  56+  :  44=  :   0-  :   78.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : LittleBeast 15 6t1024h :  100  :   0+  : 100=  :   0-  :   50.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h :  200  :  45+  : 153=  :   2-  :   60.75%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Vafra 14.12.1 6t1024h :  100  :  24+  :  76=  :   0-  :   62.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Vafra 14.12.2 6t1024h :  100  :  10+  :  90=  :   0-  :   55.0%
ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Vanilla 15 6t1024h :  100  :   2+  :  97=  :   1-  :   50.5%

ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Brainlearn 29 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h :  200  :  13+  : 166=  :  21-  :   48.0%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Lc0.cuda 0.31.1 611246.384 6t1024h :  100  :  10+  :  77=  :  13-  :   48.5%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : ShashChess 35.3 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h :  200  :   2+  : 153=  :  45-  :   39.25%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Stockfish 11 6t1024h :  100  :  38+  :  51=  :  11-  :   63.5%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Stockfish 12 6t1024h :  100  :  13+  :  82=  :   5-  :   54.0%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Stockfish 13 6t1024h :   99  :   3+  :  77=  :  19-  :   41.92%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Stockfish 14 6t1024h :  100  :   5+  :  82=  :  13-  :   46.0%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Stockfish 14.1 6t1024h :  100  :   5+  :  80=  :  15-  :   45.0%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Stockfish 15.1 6t1024h :  100  :   2+  :  84=  :  14-  :   44.0%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Vafra 14.12.2 6t1024h :  100  :  12+  :  77=  :  11-  :   50.5%
ShashChess 36 MCTS.1-PL.On-SS.AllOn-LiveBook.On.D255V0 6t1024h : Vanilla 15 6t1024h :  100  :   4+  :  81=  :  15-  :   44.5%