ChatGPT was reportedly defeated at chess by a 50-year-old chess program. An engineer recently pitted the latest ChatGPT4o model against an Atari 2600 chess engine on the beginner difficulty level, resulting in the defeat of the advanced AI model.

ChatGPT vs. Atari 2600

According to MSN, Chess engines have led human players in the ancient game since 1997, when IBM’s Deep Blue supercomputer defeated the world chess champion, Gary Kasparov. Modern Stockfish models have an estimated chess rating or ELO of over 3,600, whereas the best human chess players have only reached a 2,800 ELO.

Though ChatGPT and Atari are significantly below those ratings, the matchup is interesting since an older model defeated a very advanced modern model.

The experiment began when Citrix Engineer Robert Caruso conversed with ChatGPT about the history of chess in an attempt to see “how quickly” the model could beat a computer that can only think one or two moves ahead. He used an emulation of Atari’s 1979 Video Chess cartridge and asked ChatGPT to analyze board positions and decide its next moves based on images.

“ChatGPT got absolutely wrecked on the beginner level,” Caruso said in his LinkedIn post. “Despite being given a baseline board layout to identify pieces, ChatGPT confused rooks for bishops, missed pawn forks, and repeatedly lost track of where pieces were—first blaming the Atari icons as too abstract to recognize, then faring no better even after switching to standard chess notation.”

Even with Caruso stopping ChatGPT from making some extremely poor moves, the cutting-edge large language model was forced to concede after an hour and a half match. Though it’s noteworthy that ChatGPT isn’t designed to be a chess computer, this does raise interesting questions about AI advancements and their current capabilities of adapting and operating outside of their given functions.