Written by

Bernard Marr

Bernard Marr is a world-renowned futurist, influencer and thought leader in the fields of business and technology, with a passion for using technology for the good of humanity. He is a best-selling author of 20 books, writes a regular column for Forbes and advises and coaches many of the world’s best-known organisations. He has over 2 million social media followers, 1 million newsletter subscribers and was ranked by LinkedIn as one of the top 5 business influencers in the world and the No 1 influencer in the UK.

Bernard’s latest book is ‘Business Trends in Practice: The 25+ Trends That Are Redefining Organisations’

View Latest Book

Artificial Intelligence: The Clever Ways Video Games Are Used To Train AIs

2 July 2021

Who says you can’t get smart playing video games? Although the idea of spending hours playing video games isn’t usually recommended for humans to increase their intelligence, the realistic 3-D graphics and environments of many video games just might make video games the perfect learning tool for artificial intelligence.

The data problem

AI algorithms get smarter and learn to perform tasks by being fed enormous amounts of data. When you’re on Facebook, this doesn’t present a huge obstacle. Facebook creates huge data sets daily and also has the financial capability to close any gaps. There are millions of photographs on Facebook that are already labelled which then helps its AI algorithm know who to tag on future images. But aside from huge data-generating companies, the majority of companies don’t acquire the volumes of data required to properly train AI algorithms.

In addition, humans just don’t have the time and patience to spend teaching AI algorithms EVERYTHING they need to know. But video games have patience and time in abundance. 

Assassin’s Creed inspires computer scientist to use video games to train AI

When Adrien Gaidon, a computer scientist at Xerox Research centre Europe, saw the trailer for the video game Assassin’s Creed he was fooled into thinking it was a trailer for a movie because of its realistic look. When he realised it was actually computer-generated imagery (CGI), he thought if he could be fooled into thinking video games were real, perhaps AI algorithms could be too. 

Gaidon and his team used Unity, a widely used game development engine for 3-D video games, to create scenes to help train deep-learning algorithms. Not only did they create synthetic environments, but they imported a real scene into the virtual world. This allows them to compare the effectiveness of training algorithms with virtual environments against those trained by real images. His testing continues.

Microsoft’s Project Malmo uses Minecraft

Project Malmo is an open-source platform that allows AI experiments within the world of Minecraft that was created to support fundamental research into AI. Although you might wonder how Minecraft’s blocky interpretation of the world translates into learning how the real world works, its power for training comes into play because players of Minecraft are submerged into that world and interact with the Minecraft environment and perceive it through senses. This can be a critical training ground for AI before it’s unleashed into the real world.

Since Minecraft offers its users endless opportunities for simple and more complex tasks. This platform could also help researchers understand the building blocks of intelligence and how AI interacts with its world, learn how it makes sense of its surroundings and develops an internal representation of an environment. Since computers view the world in an entirely different way, we might be surprised by the outcome. A computer’s perspective of the world might feel entirely foreign to us, but nonetheless, its perspective was formed using the same variables as humans had to interpret the world around us. 

Grand Theft Auto instrumental in autonomous cars 

Artur Filipowicz and his team at Princeton University used Grand Theft Auto to help its AI algorithm learn about stop signs. The first job was to train the AI to learn variations of stop signs including knowing they were stop signs when they were partially obscured, in the shadows, dirty with mud or covered in snow, represented at different times of day and more. Instead of sourcing all the possible images or going out to shoot their own images of stop signs in a variety of circumstances, the team used Grand Theft Auto V as a training ground. The game had a plethora of stop signs in a variety of situations that were ideal for training AI. In order for the game to be used as a driving simulator for AI, it needed to be tweaked so it could be played by another computer programme instead of only by a human. 

Several research groups are now using the game to train AI algorithms that could be put to use helping a self-driving car navigate in the real world. 

Virtual worlds allow for customization and quick changes

A team from Intel Labs and Darmstadt University created a software layer between Grand Theft Auto V and a computer’s hardware that automatically classifies different objects from scenes in the game. These labels are then fed into a machine-learning algorithm to help train the system. This automatic annotation of the world is done significantly quicker and more comprehensively than could be achieved by humans. 

The beauty of virtual worlds is the ability to make whatever you want—things that would be time-consuming and expensive in the real world. Virtual worlds allow scalability that isn’t feasible when only humans are charged with the task, at least on a timeline that is reasonable. 

The very realistic video games that are commonplace today and the flexibility they provide make them an ideal training ground for AI algorithms that require abundant data to learn from. Next time you’re ready to dismiss the credibility of video games, think twice. They are proving very worthwhile in AI training. 

Data Strategy Book | Bernard Marr

Related Articles

How Do We Use Artificial Intelligence Ethically | Bernard Marr

How Do We Use Artificial Intelligence Ethically?

I’m hugely passionate about artificial intelligence (AI), and I'm proud to say that I help companies use AI to do amazing things in the world [...]

How Artificial Intelligence Can Help Small Businesses | Bernard Marr

How Artificial Intelligence Can Help Small Businesses

Small and medium-sized businesses all over the world are benefiting from artificial intelligence and machine learning – and integrating AI into core business functions and processes is getting more accessible and more affordable every day. [...]

What Really Is The Tesla Bot And How Much Will It Cost | Bernard Marr

What Really Is The Tesla Bot And How Much Will It Cost?

Elon Musk has just announced that Tesla will begin developing a humanoid robot called the Tesla Bot that is designed to perform “unsafe, repetitive, or boring” tasks. [...]

Should I Choose Machine Learning or Big Data | Bernard Marr

Should I Choose Machine Learning or Big Data?

Big Data and Machine Learning are two exciting applications of technology that are often mentioned together in the space of the same breath [...]

What Is The Next Level Of AI Technology | Bernard Marr

What Is The Next Level Of AI Technology?

Artificial Intelligence (AI) has permeated all aspects of our lives – from the way we communicate to how we work, shop, play, and do business. [...]

The 7 Biggest Ethical Challenges of Artificial Intelligence | Bernard Marr

The 7 Biggest Ethical Challenges of Artificial Intelligence

Today, artificial intelligence is essential across a wide range of industries, including healthcare, retail, manufacturing, and even government. [...]

Stay up-to-date

  • Get updates straight to your inbox
  • Join my 1 million newsletter subscribers
  • Never miss any new content

Social Media



View Podcasts