What looks like random behavior turns out to be the AI taking full advantage of a bug nobody knew existed in Q*bert.
In the days before patching games after launch was a thing, if a new game contained a bug it didn’t get fixed, or at least not until the next batch of cartridges were manufactured. Some of those bugs turned out to double as a way of cheating, and an AI trained to play Atari games just discovered a new cheat-friendly bug in Q*bert.
As Techspot explains, a team of researchers at the University of Freibeurg in Germany were using the Gym toolkit to train and test an artificial intelligence playing the classic action puzzle game Q*bert on Atari. This was a game originally released in the arcades in 1982 and then on the Atari 2600 first in 1983, so there’s been plenty of time to discover all the bugs and cheats contained in the game. One was missed, though.
As the video above shows, at the 20 second mark the AI completes the first level of the game and then seems to randomly start jumping around. It isn’t until you look to the top of the screen and see the score that you realize this isn’t random behavior. The researchers believe the AI discovered a new bug in the game and worked out it could exploit it to drastically increase its score.
The researchers haven’t worked out yet how the AI managed to do this. One of the original creators of Q*bert for the arcade version, Warren Davis, commented on Twitter that this bug “doesn’t look right,” suggesting it may be exclusive to the Atari 2600 version.
The AI uses machine learning to train and play different games. In this case an Evolution Strategy algorithm was used, which allowed the AI to train and start using novel strategies to beat the game in just five hours. What wasn’t expected was one of those novel strategies discovering a new cheat in a 35-year-old game.