My experience with Udacity Deep Learning Nanodegree

This week, I finished Udacity Deep Learning Nanodegree (DLND) which I started November 2017. I will write about my experience with the course. And I hope it will help those who are considering to take the course decide to enroll.

TL;DR

• Udacity Deep Learning Nanodegree (DLND) is good for those who don't have DL background to understand and experience DL in reasonable period.
• The projects in the course and the review system are very helpful for me to use DL technics rather than just knowing them.
• It is worth paying the tuition fee $599 and spending 4 months (12 hrs/week) of your time.

Background

I am a software engineer with 7+ years of experience of new service development. so when I was considering to take this course, I didn't worry about coding that much. but I didn't have professional expertise in ML/DL.

I was really interested with deep learning, and personally built an object detection model with tensorflow's API as I try a new library or a programming language before taking the course.
But it was difficult for me to improve the model by tuning hyper parameter, customizing architecture without fundamental understanding of the algorithm behind the high level API.

I also wanted to know about what we can do with DL comprehensively.
So taking an online course seemed good choice to me, and it was.

About Udacity Deep Learning Nanodegree

DNLD is one of paid courses at Udacity, which has real projects with strict deadlines scheduled to finish all projects in 4 months. you have to satisfy requirements to pass strict reviews which are done manually.
Before enrolling it, I thought the tuition fee $599 was bit expensive. but it turned that it's worth paying it.

Curriculum

I will write about what I learned in the curriculum. it starts with introduction and the basics of neural networks and goes to advanced models.

Introduction

The first part is about the demos of DL applications, development environments (anaconda and jupyter notebook) and a math refresher relevant to DL.
You can preview this part for free.

Neural Networks

The 2nd part is about the basics of neural networks. you learn and implement simple neural network. you also learn how to use Keras and Tensorflow.

Project: Predicting Bike Sharing Data In this project, you implement a simple neural network with numpy. this was a good exercise to understand how forward path and back propagation work in neural network.

Convolutional Neural Networks

Convolutional Neural Networks (CNNs) have outperformed traditional ML models in computer vision tasks. It extracts features from images to understand high level concepts of them. You learn different CNN architectures, and transfer learning by implementing mini projects.

I enjoyed skin cancer detector project which wasn't easy to get good results. so I read related articles and advices posted in Slack to deepen my understanding.

Project: Dog Breed Classifier In this project, you implement a CNN with Keras to classify dog breed.

Recurrent Neural Networks

Recurrent Neural Networks (RNNs) act on sequences of inputs keeping context, such as sentences and voice sounds. Long short term memory networks (LSTMs) is a special kind of RNN and is capable of long term dependencies.

You implement both RNN and LSTM. And you also learn word2vec which is useful to represent words as vectors to get conceptual meanings of words.

Project: Generate TV Scripts In this project, you generate TV scripts from existing scripts dataset using RNN.

Generative Adversarially Networks

Generative Adversarial Networks (GANs) are unique networks compared to CNNs and RNNs. In a GAN, there is a generator network and a discriminator network. while the generator is trying to generate fake images, the discriminator is trying to discriminate if it's fake. Both networks get trained adversarially.

Project: Generate Faces In this project, you generate human face images using GANs. This network was complicated to build and train. so I checked related articles to get good results. it was actually harder for me to finish than the above projects.

Deep Reinforcement Learning

Reinforcement Learning is a type of machine learning where the agent learns how to maximize its performance by interacting the environment. Google's Deep Mind is famous in this area. they built game play agents which are remarkably better at each game than human.

You learn about reinforcement learning first and then learn how deep neural networks make it better. This is, for me, very big and complicated topic. and the goal is different from the other models in this course.

Project: Teach a quadcopter how to fly This is the last project of this course. you implement a quadcopter agent which learns how to fly using deep deterministic policy gradient (DDPG). This was the hardest project for me to complete. I read related articles as well as original papers. finally I was able to converge the training and my quadcopter successfully flied in a simulation environment as I expected.

Sum up

I couldn't write all of what I learned in the course since they are a lot. After I finished the course, or even while proceeding, I though that the course was well organized and I would have spend much more time if I learned them myself.
So I highly recommend this course people who want to learn what DL is and what DL is good at, how difficult to build models. the tuition fee was much cheaper than the saved time.

Even though I'm still not an expert, I have better understanding about DL. it's good for me as a software engineer because I will be able to choose DL as a solution for some problems.

Thank you for reading this article. I hope you enjoyed it. :)