I took CS399 Independent Study w. Prof. Mykel Kochenderfer. I worked on the intersection problem in which a car is at the bottom of a T-intersection and wants to make a left turn while avoiding collision with another car crossing the intersection. The problem was first modeled as a simple discrete MDP and solved using value iteration implemented in Julia. Then, because I wanted to explore DQN techniques, I coded the problem into a simple video game using mtTkinter and then switching to the PyGame framework. The code was made to be similar to an OpenAi Gym environment in that it has a step function that executes an action and then returns the state, rewards and terminal boolean. I used Tensorflow to code the convolutional neural network using the DQN algorithm with experience replay. The CNN architecture was a scaled down version of what was used in the Deep Mind Atari paper, ut with the same dimensions, e.i. 80x80x4. An example of a training session is below.