Who says you can’t get smart playing video games? Although the idea of spending hours playing video games isn’t usually recommended for humans to increase their intelligence, the realistic 3-D graphics and environments of many video games just might make video games the perfect learning tool for artificial intelligence.

The data problem

AI algorithms get smarter and learn to perform tasks by being fed enormous amounts of data. When you’re on Facebook, this doesn’t present a huge obstacle. Facebook creates huge data sets daily and also has the financial capability to close any gaps. There are millions of photographs on Facebook that are already labelled which then helps its AI algorithm know who to tag on future images. But aside from huge data-generating companies, the majority of companies don’t acquire the volumes of data required to properly train AI algorithms.

In addition, humans just don’t have the time and patience to spend teaching AI algorithms EVERYTHING they need to know. But video games have patience and time in abundance. 

Assassin’s Creed inspires computer scientist to use video games to train AI

When Adrien Gaidon, a computer scientist at Xerox Research centre Europe, saw the trailer for the video game Assassin’s Creed he was fooled into thinking it was a trailer for a movie because of its realistic look. When he realised it was actually computer-generated imagery (CGI), he thought if he could be fooled into thinking video games were real, perhaps AI algorithms could be too. 

Gaidon and his team used Unity, a widely used game development engine for 3-D video games, to create scenes to help train deep-learning algorithms. Not only did they create synthetic environments, but they imported a real scene into the virtual world. This allows them to compare the effectiveness of training algorithms with virtual environments against those trained by real images. His testing continues.

Microsoft’s Project Malmo uses Minecraft

Project Malmo is an open-source platform that allows AI experiments within the world of Minecraft that was created to support fundamental research into AI. Although you might wonder how Minecraft’s blocky interpretation of the world translates into learning how the real world works, its power for training comes into play because players of Minecraft are submerged into that world and interact with the Minecraft environment and perceive it through senses. This can be a critical training ground for AI before it’s unleashed into the real world.

Since Minecraft offers its users endless opportunities for simple and more complex tasks. This platform could also help researchers understand the building blocks of intelligence and how AI interacts with its world, learn how it makes sense of its surroundings and develops an internal representation of an environment. Since computers view the world in an entirely different way, we might be surprised by the outcome. A computer’s perspective of the world might feel entirely foreign to us, but nonetheless, its perspective was formed using the same variables as humans had to interpret the world around us. 

Grand Theft Auto instrumental in autonomous cars 

Artur Filipowicz and his team at Princeton University used Grand Theft Auto to help its AI algorithm learn about stop signs. The first job was to train the AI to learn variations of stop signs including knowing they were stop signs when they were partially obscured, in the shadows, dirty with mud or covered in snow, represented at different times of day and more. Instead of sourcing all the possible images or going out to shoot their own images of stop signs in a variety of circumstances, the team used Grand Theft Auto V as a training ground. The game had a plethora of stop signs in a variety of situations that were ideal for training AI. In order for the game to be used as a driving simulator for AI, it needed to be tweaked so it could be played by another computer programme instead of only by a human. 

Several research groups are now using the game to train AI algorithms that could be put to use helping a self-driving car navigate in the real world. 

Virtual worlds allow for customization and quick changes

A team from Intel Labs and Darmstadt University created a software layer between Grand Theft Auto V and a computer’s hardware that automatically classifies different objects from scenes in the game. These labels are then fed into a machine-learning algorithm to help train the system. This automatic annotation of the world is done significantly quicker and more comprehensively than could be achieved by humans. 

The beauty of virtual worlds is the ability to make whatever you want—things that would be time-consuming and expensive in the real world. Virtual worlds allow scalability that isn’t feasible when only humans are charged with the task, at least on a timeline that is reasonable. 

The very realistic video games that are commonplace today and the flexibility they provide make them an ideal training ground for AI algorithms that require abundant data to learn from. Next time you’re ready to dismiss the credibility of video games, think twice. They are proving very worthwhile in AI training.