Reinforcement Learning Projects

This repository contains implementations of various Reinforcement Learning algorithms to solve classic control and navigation tasks. The projects demonstrate the application of both tabular methods and Deep Reinforcement Learning (DRL) developed during my MSc studies.

Projects

1. Flappy Bird Agent

Goal: Train an autonomous agent to play Flappy Bird.

• Algorithms: Q-Learning, SARSA (Tabular methods).
• Environment: FlappyBird-v0 (via PyGame Learning Environment & OpenAI Gym).
• Key Implementations:
  • State Preprocessing: Custom discretization and normalization of the continuous 8-dimensional state space (bird position, velocity, pipe distances) into a compact representation suitable for tabular learning.
  • Reward Shaping: Engineered reward functions to accelerate learning, including potential-based shaping based on the bird's vertical and horizontal distance to the next pipe gap.
  • Environment Wrapping: Custom Game wrapper inheriting from gym.Env to integrate preprocessing and reward shaping pipelines directly into the environment step.

2. MiniGrid Navigation

Goal: Solve visual navigation duties involving sparse rewards and object interactions (Door & Key).

• Algorithms:
  • PPO (Proximal Policy Optimization): Implemented an Actor-Critic architecture with Generalized Advantage Estimation (GAE) and clipped surrogate objective.
  • DDQN (Double Deep Q-Network): Implemented for stable off-policy value learning.
• Environment: MiniGrid (e.g., MultiRoom-N6-v0).
• Key Implementations:
  • Visual Perception: Custom CNN architecture (CNNBase) designed to process grid-world image observations (RGB output).
  • Heuristic Reward Shaping: Complex reward logic handling sub-goals such as approaching closed doors, successfully toggling/opening doors, and navigation towards the goal, mitigating the sparse reward problem.
  • Modular Design: Abstract BaseAgent class with concrete PyTorch implementations for PPO and DDQN agents.

Technologies Used

• Core: Python 3.x
• Deep Learning: PyTorch
• Environments: Gymnasium, MiniGrid, PyGame Learning Environment (PLE)
• Data & Visualization: NumPy, Matplotlib, ImageIO

How to Run

The projects are provided as self-contained Jupyter Notebooks, optimized for execution in Google Colab but compatible with local Jupyter environments.

1. Flappy Bird: Open FlappyBird- mid assignment QLEARNING& SARSA/FlappyBirdMidTerm.ipynb.
2. MiniGrid: Open MiniGrid-final assignment PPO & DDQN/MiniGrid implementation.ipynb.

Note: Each notebook includes an installation cell at the top to set up the required dependencies (video drivers, environment packages).