Q-Learning Agent in VR

1. Introduction

An AI agent is often not tangible. It’s an artificially being that usually resides in virtual space. We can seldom see robots and drones with AI on media, but not usually in the physical world. It’s very hard to bring agents to the physical world, but if we let them stay in their space, it becomes a lot easier to interact. The VR technologies let us be in a virtual space, thus we can experience the embodiment of an AI agent. This project is motivated by this gateway to space of another being.

2. Problem

It’s very costly to bring AI to physical space, and it would cost a tremendous amount to develop and test such agent in physical space. But, if we are able to simulate good enough for AI agents, we believe that the cost of development can be reduced drastically. And, we believe the AI agent in VR can be very entertaining.

3. Practices

For comfortable experience, OBP suggests few things. And, these are the main guideline that we will follow:
(i) distance to focusing object is between 0.75 and 3.5 meters [OBP 10],
(ii) post-processing effects are applied to both eyes [OBP 10],
(iii) use default FOV [OBP 13],
(iv) Viewing the environment from a stationary position is most comfortable. [OBP 6] Users will mostly be standing in a training-stage, and evaluation stage in a game form would also mostly be in a stationary position.

4. Rules for Agents

All types of agents have 30 decisions/sec and moves in same speed except human players. When the ball bounces, its reflection angle and speed randomly change, so that its harder for the AI agents. Without continuous random adjustments, Q-Agents can play the game for very long time. It's designed to test their limits.

5-1 Q-Learning Agents Playing each other

Most of the mistakes are by movements that are off by an inch. The high ball speed can reduce the game time effectively.

5-2 Q-Agent vs Greedy Agent

A greedy agent can never beat a Q-Learning agent, even though they have same decision making rates and moving speed.

Acknowledgement

This project was advised by Prof. Eric Shaffer.

• AI, Physics, and Graphics Programming by Byung Il Choi
• Graphics Programming and Game Play by Seongsu Ha
• User Interface and Graphic Design by Chris Wegenek
• Sound and Testing by Alejandro Marin