Project 1: Navigation

Introduction

This project is meant to respond to the requirements of the Udacity Deep Reinforcement Learning Project 1: Navigation.

The banana environment is set up for the student and looks like the image above when rendered.

A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana. Thus, the goal of your agent is to collect as many yellow bananas as possible while avoiding blue bananas.

The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction. Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:

• 0 - move forward.
• 1 - move backward.
• 2 - turn left.
• 3 - turn right.

The task is episodic, and in order to solve the environment, your agent must get an average score of +13 over 100 consecutive episodes.

Getting Started

Dependencies

1. Create (and activate) a new environment with Python 3.6.

   • Linux or Mac:

   conda create --name drlnd python=3.6
   source activate drlnd

   • Windows:

   conda create --name drlnd python=3.6 
   activate drlnd

2. Follow the instructions in this repository to perform a minimal install of OpenAI gym.

3. Clone the repository (if you haven't already!), and navigate to the python/ folder. Then, install several dependencies.

git clone https://github.com/udacity/deep-reinforcement-learning.git
cd deep-reinforcement-learning/python
pip install .

4. Create an IPython kernel for the drlnd environment.

python -m ipykernel install --user --name drlnd --display-name "drlnd"

5. Before running code in a notebook, change the kernel to match the drlnd environment by using the drop-down Kernel menu.

6. Refer to Navigation.ipynb.

If you are running on a machine without a display, you will need to set a no_graphics = True flag when the environment is instantiated, but this will make the output pretty boring.

If you'd like to see the untrained agent blunder around the environment, un-comment the code under section 3.

The first cell of code in section 4.1 is where we train the agent with hyper parameters previously determined through manually blundering around with the or running the loop in 4.2. This setup meets the assignment criteria of +13 reward averaged over 100 episodes.

Finally, you can watch the trained agent running parameters from 4.1.