It was another defining moment in computing, then, when AlphaGo -- a Go-playing AI developed by Google subsidiary DeepMind -- beat professional player Fan Hui in late 2015, followed by world champion Lee Sedol the subsequent spring. Not only did AlphaGo win both encounters convincingly but showed creativity in its play that surprised even the most seasoned grandmasters of the game. Some of the move sequences AlphaGo has played have actually influenced how humans approach the game at the highest level.
That's particularly important because it makes the idea of artificial intelligence moving beyond brute-force calculations easier to grasp. While a research paper may be nigh on impossible to understand to anyone outside the field, the use of wholly unconventional moves in match play is a clear indicator AlphaGo has outgrown human tuition. Indeed, the AI was initially trained on a vast dataset of recorded Go matches with which it gained a predictive capacity -- a knowledge of likely next moves based on the current makeup of the board.
The rest of its game sense, however, was accumulated more organically, by playing matches against itself. Thousands upon thousands of matches, in fact, and as it improved, so did its opponent. A lifetime of games -- more practice than any human could hope to process and convert into applicable experience.
The focus for subsequent iterations of AlphaGo has been on making the system more efficient and minimizing human coaching, whilst preserving the AI's unbeaten record against top professionals. AlphaGo Zero, an AI trained exclusively against itself with the basic rules as its only reference point, surpassed all previous designs after 40 days of training. Building upon this, successor AlphaZero mastered chess, shogi and Go after just hours of training, to the point of being able to beat the best computer competitors in these respective games (including AlphaGo Zero).
Before DeepMind went about creating the best Go player that's ever existed, it worked on an algorithm that learned to play Atari 2600 games on its own. Video games have become a very popular tool in AI research. Understandably so, as they offer not a rigid board game with clear rules but a complex virtual environment and safe space for AIs to develop through reinforcement learning -- trial and error (but more nuanced). What's particularly impressive about AIs playing video games is that they are seeing the screen like you or I (as opposed to reading lines of code for information on game state), and trying to make sense of the mess of pixels to figure out mechanics and win conditions.
Observing computers playing computer games has resulted in some interesting, unintended behavior. There are many stories of AIs finding previously undiscovered bugs, glitches and skips in games when provoked with a simple reward trigger like a high score or fastest average speed in a racing game. A DeepMind researcher continues to compile a list of these (and other AI phenomena), and just one example is an AI discovering how to clip through walls in a Sonic game in an effort to finish levels as fast as possible to maximize its score. This AI was competing in the OpenAI Retro Contest ran by the research organization co-founded by Elon Musk.
AIs seem to be getting so good at video games that Unity Technologies has developed a title specifically to test the limits of reinforcement learning. Obstacle Tower is a 3D, puzzle-ridden platformer with procedurally generated levels that ensure an AI can't be programmed to carry out a specific set of actions and must, instead, understand the puzzle to complete the level. Currently, participants have access to the first 25 stages, but in mid-April, all 100 floors of the tower will be unlocked for the ultimate challenge. The game is far from easy, too, with each level increasing in difficulty and human players tapping out at around level 15.
In recent years, there have been several notable competitive AI gamers, from MIT's competent Super Smash Bros. Melee fighter to DeepMind's Quake III Arena capture-the-flag specialists, which are better teammates than actual humans. Perhaps the biggest achievement to date, though, was AIs wiping the floor with StarCraft II professionals earlier this year.
StarCraft II is arguably the hardest video game to play at the pro level. It's a fast-paced, real-time strategy in which the player must juggle resource and infrastructure management, controlling multiple units and tactical decision-making, often with incomplete information of what the opponent is up to. The game is as complex as it is unforgiving, with the possibility of one seemingly minor mistake snowballing into a huge, insurmountable disadvantage.