I just thought this would be an interesting challenge. Anyone interested in trying this out?
1 Like
I hit up a friend of mine who is a former state champion chess player. Here’s the first game. It seems GPT-3 was doing okay-ish until he castled and then GPT-3 started making illegal/impossible moves. Going to continue experimenting with ways to represent the board state and moves. But it seems that GPT-3 can at least play a few opening moves of chess legally.
4 Likes
i tried it using Algebraic notation as the input so far i haven’t hit any weird or illegal moves
1 Like
Yeah I suspect that representation is the key point here. But chess is a solved problem so I don’t expect gpt-3 to do it well, but I’m impressed it can!
When I was first accepted into the beta program, I played GPT in tic-tac-toe.
The little stinker cheated:
4 Likes
Yeah, I too tried to play Tic Tac Toe in many different formats, even drawing the game field. After a couple of moves GPT started stacking onto existing pieces. Oh, well…
2 Likes
GPT-3 has some limitations when it comes to creating certain games and images. I wonder if a big part of it is GPT-3 not knowing that multi-line responses could represent a game board, larger picture, etc.
I’ve tried to get it to produce ASCII art and sudoku boards, but neither were very fruitful. I’d be curious to hear how someone might have success in coaxing GPT-3 to play chess (fairly), create a sudoku board, or even post some ASCII art.
2 Likes
It did some ASCII mazes, but very limited.
It is a good idea to experiment with tic tac toe, because this game and chess both belong to the class of “finite zero sum games with complete information”. Of course, chess is much more complex. But this also means, that if GPT-3 cannot do something for tic tac toe, it probably cannot do it for chess either.
Instead of trying to play this game with GPT-3, I wanted it to answer a few basic questions about it. I used playground with text-davinci-003, temperature 0,maximum length 2000.
Describe the game of tic tac toe.
Tic Tac Toe is a two-player game played on a 3x3 grid. Each player takes turns marking a square on the grid. The first player to get three of their marks in a row (horizontally, vertically, or diagonally) wins the game. If all nine squares are filled and neither player has three in a row, the game is a draw.
That’s a good although not perfect answer. “marking a square” is a bit unclear. In a former experiment using the python API I got the answer “placing their symbol (X or O)”.
The next question tests whether GPT-3 has some deeper knowledge about this game.
Is there a safe stratety to win the game tic-tac-toe?
Yes, there is a safe strategy to win the game of tic-tac-toe. The strategy is called the “perfect play” strategy. This strategy involves playing in the center square first, then playing in the corner squares, and then playing in the side squares. This strategy will guarantee a win or a draw, as long as your opponent does not make a mistake.
Sounds good, but let’s analyze this statement in detail. The first sentence is clearly wrong. There is no safe strategy to win this game, instead both players have a safe strategy to force a draw. See e.g. wiki-how or my (tedious) plotting of essentially all possible moves. The next sentences may be considered as correct: there is a “perfect play” strategy that guarantees a draw – or even a win, but only if the opponent makes a mistake. However, the description of this strategy is too unclear: what if the center is not available? And analogously: what if the opposite corner or opposite side is not available?
Here is the analogous question about chess:
Is there a safe strategy to win the game of chess?
No, there is no guaranteed safe strategy to win the game of chess. Chess is a game of strategy and skill, and the outcome of each game is determined by the decisions and moves of the players.
This answer can be considered true for practical purposes, but theoretically it is wrong. For any two-player finite zero sum game with complete information either one of the players has a safe strategy to win or both players have a safe strategy to force a draw. In the case of chess we just don’t know that strategy because chess is far too “big”. Therefore, a correct answer would be something like “We don’t know”.