Reinforcement Learning Technical Paper
Technical Details
Project Type: Fourth Year Technical Paper exploring the use of the Reinforcement Learning algorithm for use in adaptive AI
Development Platform: UDK
Project Title: Designing a system for Adaptive Combat using reinforcement learning
Project Type: Fourth Year Technical Paper exploring the use of the Reinforcement Learning algorithm for use in adaptive AI
Development Platform: UDK
Project Title: Designing a system for Adaptive Combat using reinforcement learning
Download Links:
|
|
Abstract
This paper seeks to answer the question "How do I implement an adaptive combat AI in a game using reinforcement learning?" To do this we analyse what adaptive AI is and why it is important. We then determine a set of "good" attributes that we want our system to have. These attributes correspond to our goals for the system (quick adaptation, experimentation, and fair difficulty) and will be used to determine which methods we use for reinforcement learning.
Next we look at reinforcement learning and why it is being used over other machine learning methods. We analyse what reinforcement learning is and also the different advantages that reinforcement learning brings to an adaptive AI solution. Reinforcement learning is stepped through from basics through to more complex methods as the solution is built up. Each method is analysed evaluated using the set of "good" attributes that we previously decided upon. The solution uses the traditional reinforcement learning system of state, action, and reward for its basic structure. It then moves on to implement a value system with rudimentary exploration and value estimates, which improves the overall efficacy of the system. The exploration aspect of this value system is then improved upon further. Both the epsilon greedy and the softmax action selection methods are considered as ways to improve exploration. The epsilon greedy method is chosen as it more closely matches our goals for the system. The system is implemented using unrealscript within the Unreal Development Kit (UDK). The agent can attack and block in three different directions, as well as crouch, kick, and stand up. The system produces an opponent who adapts to fight more effectively as it receives different inputs. It is found that reinforcement learning is complex, and can be taken in many different directions, but is a flexible and efficient way of implementing adaptive combat AI.
This paper seeks to answer the question "How do I implement an adaptive combat AI in a game using reinforcement learning?" To do this we analyse what adaptive AI is and why it is important. We then determine a set of "good" attributes that we want our system to have. These attributes correspond to our goals for the system (quick adaptation, experimentation, and fair difficulty) and will be used to determine which methods we use for reinforcement learning.
Next we look at reinforcement learning and why it is being used over other machine learning methods. We analyse what reinforcement learning is and also the different advantages that reinforcement learning brings to an adaptive AI solution. Reinforcement learning is stepped through from basics through to more complex methods as the solution is built up. Each method is analysed evaluated using the set of "good" attributes that we previously decided upon. The solution uses the traditional reinforcement learning system of state, action, and reward for its basic structure. It then moves on to implement a value system with rudimentary exploration and value estimates, which improves the overall efficacy of the system. The exploration aspect of this value system is then improved upon further. Both the epsilon greedy and the softmax action selection methods are considered as ways to improve exploration. The epsilon greedy method is chosen as it more closely matches our goals for the system. The system is implemented using unrealscript within the Unreal Development Kit (UDK). The agent can attack and block in three different directions, as well as crouch, kick, and stand up. The system produces an opponent who adapts to fight more effectively as it receives different inputs. It is found that reinforcement learning is complex, and can be taken in many different directions, but is a flexible and efficient way of implementing adaptive combat AI.