Playing Doom with Anticipator-A3C Based Agents Using Deep Reinforcement Learning and the ViZDoom Game-AI Research Platform

The built-in game agents act according to the pre-written scripts and make decisions, take actions like they have been stated. They acquire and take advantage of unfair information, instead of acting flexibly like human players, who make decisions only based on game screens. This chapter focuses on studying the application of Deep Learning and Reinforcement Learning in games agents and the improvement of the related algorithms. The goal is to develop a game agent that makes decisions in human’s way and gets rid of relying on unfair information. A game agent (CNN) is implemented by augmenting the A3C algorithm. This agent takes the original real-time game screen as the input of the network, and then output the matching policy. The agent interacts with ViZDoom and reads the real-time game screen to make decisions for controlling the character to act. This chapter improved the A3C algorithm by adding an anticipator network to the original model structure. The goal of doing this is to make the agent act more like human players. It will generate anticipation before making decisions, then combine the real-time game screen with anticipation images together as a whole input of the network defined by the A3C algorithm. It can use the combination of the data to make decisions and output the discrete actions. Because the method only changes the structure of data for the input of the network, so it is a model-free method and can be easily transplanted to other algorithms. The performance of A3C is compared with variants proposed in this chapter, analyzed the differences between them and gathered the experimental data from the latest articles as a comparison which studies the same problem. The result shows, that the A3C algorithm with Anticipation performs better than the A3C algorithm.