I just thought this would be an interesting challenge. Anyone interested in trying this out?
1 Like
I hit up a friend of mine who is a former state champion chess player. Hereâs the first game. It seems GPT-3 was doing okay-ish until he castled and then GPT-3 started making illegal/impossible moves. Going to continue experimenting with ways to represent the board state and moves. But it seems that GPT-3 can at least play a few opening moves of chess legally.
4 Likes
i tried it using Algebraic notation as the input so far i havenât hit any weird or illegal moves
1 Like
Yeah I suspect that representation is the key point here. But chess is a solved problem so I donât expect gpt-3 to do it well, but Iâm impressed it can!
When I was first accepted into the beta program, I played GPT in tic-tac-toe.
The little stinker cheated:
4 Likes
Yeah, I too tried to play Tic Tac Toe in many different formats, even drawing the game field. After a couple of moves GPT started stacking onto existing pieces. Oh, wellâŚ
2 Likes
GPT-3 has some limitations when it comes to creating certain games and images. I wonder if a big part of it is GPT-3 not knowing that multi-line responses could represent a game board, larger picture, etc.
Iâve tried to get it to produce ASCII art and sudoku boards, but neither were very fruitful. Iâd be curious to hear how someone might have success in coaxing GPT-3 to play chess (fairly), create a sudoku board, or even post some ASCII art.
2 Likes
It did some ASCII mazes, but very limited.
It is a good idea to experiment with tic tac toe, because this game and chess both belong to the class of âfinite zero sum games with complete informationâ. Of course, chess is much more complex. But this also means, that if GPT-3 cannot do something for tic tac toe, it probably cannot do it for chess either.
Instead of trying to play this game with GPT-3, I wanted it to answer a few basic questions about it. I used playground with text-davinci-003, temperature 0,maximum length 2000.
Describe the game of tic tac toe.
Tic Tac Toe is a two-player game played on a 3x3 grid. Each player takes turns marking a square on the grid. The first player to get three of their marks in a row (horizontally, vertically, or diagonally) wins the game. If all nine squares are filled and neither player has three in a row, the game is a draw.
Thatâs a good although not perfect answer. âmarking a squareâ is a bit unclear. In a former experiment using the python API I got the answer âplacing their symbol (X or O)â.
The next question tests whether GPT-3 has some deeper knowledge about this game.
Is there a safe stratety to win the game tic-tac-toe?
Yes, there is a safe strategy to win the game of tic-tac-toe. The strategy is called the âperfect playâ strategy. This strategy involves playing in the center square first, then playing in the corner squares, and then playing in the side squares. This strategy will guarantee a win or a draw, as long as your opponent does not make a mistake.
Sounds good, but letâs analyze this statement in detail. The first sentence is clearly wrong. There is no safe strategy to win this game, instead both players have a safe strategy to force a draw. See e.g. wiki-how or my (tedious) plotting of essentially all possible moves. The next sentences may be considered as correct: there is a âperfect playâ strategy that guarantees a draw â or even a win, but only if the opponent makes a mistake. However, the description of this strategy is too unclear: what if the center is not available? And analogously: what if the opposite corner or opposite side is not available?
Here is the analogous question about chess:
Is there a safe strategy to win the game of chess?
No, there is no guaranteed safe strategy to win the game of chess. Chess is a game of strategy and skill, and the outcome of each game is determined by the decisions and moves of the players.
This answer can be considered true for practical purposes, but theoretically it is wrong. For any two-player finite zero sum game with complete information either one of the players has a safe strategy to win or both players have a safe strategy to force a draw. In the case of chess we just donât know that strategy because chess is far too âbigâ. Therefore, a correct answer would be something like âWe donât knowâ.