Dive into Deep Learning¶
Dive into Deep Learning
An interactive deep learning book for students, engineers, and
researchers.
The contents are under revision.
We thank all the community contributors
for making this open source book better for everyone.
Contribute to the book.
Each section is an executable Jupyter notebook
You can modify the code and tune hyperparameters to get instant feedback to accumulate practical experiences in deep learning.
Mathematics + Figures + Code
We offer an interactive learning experience with mathematics, figures, code, text, and discussions, where concepts and techniques are illustrated and implemented with experiments on real data sets.
Active community support
You can discuss and learn with thousands of peers in the community through the link provided in each section.
Table of contents
- Introduction
- A Taste of Deep Learning
- Deep Learning Basics
- Linear Regression
- Linear regression implementation from scratch
- Gluon Implementation of Linear Regression
- Softmax Regression
- Image Classification Data (Fashion-MNIST)
- Softmax Regression from Scratch
- Softmax Regression in Gluon
- Multilayer Perceptron
- Implementing a Multilayer Perceptron from Scratch
- Multilayer Perceptron in Gluon
- Model Selection, Underfitting and Overfitting
- Weight Decay
- Dropout
- Forward Propagation, Back Propagation and Computational Graphs
- Numerical Stability and Initialization
- Environment
- Predicting House Prices on Kaggle
- Deep Learning Computation
- Convolutional Neural Networks
- From Dense Layers to Convolutions
- Convolutions for Images
- Padding and Stride
- Multiple Input and Output Channels
- Pooling
- Convolutional Neural Networks (LeNet)
- Deep Convolutional Neural Networks (AlexNet)
- Networks Using Blocks (VGG)
- Network in Network (NiN)
- Networks with Parallel Concatenations (GoogLeNet)
- Batch Normalization
- Residual Networks (ResNet)
- Densely Connected Networks (DenseNet)
- Recurrent Neural Networks
- Sequence Models
- Language Models
- Recurrent Neural Networks
- Text Preprocessing
- Building a Recurrent Neural Network from Scratch
- Gluon Implementation in Recurrent Neural Networks
- Back-propagation Through Time
- Gated Recurrent Unit (GRU)
- Long Short-term Memory (LSTM)
- Deep Recurrent Neural Networks
- Bidirectional Recurrent Neural Networks
- Optimization Algorithms
- Computational Performance
- Computer Vision
- Image Augmentation
- Fine Tuning
- Object Detection and Bounding Box
- Anchor Boxes
- Multiscale Object Detection
- Object Detection Data Set (Pikachu)
- Single Shot Multibox Detection (SSD)
- Region-based CNNs (R-CNNs)
- Semantic Segmentation and Data Sets
- Fully Convolutional Networks (FCN)
- Neural Style Transfer
- Image Classification (CIFAR-10) on Kaggle
- Dog Breed Identification (ImageNet Dogs) on Kaggle
- Natural Language Processing
- Word Embedding (word2vec)
- Approximate Training
- Implementation of Word2vec
- Subword embedding (fastText)
- Word Embedding with Global Vectors (GloVe)
- Finding Synonyms and Analogies
- Text Sentiment Classification: Using Recurrent Neural Networks
- Text Sentiment Classification: Using Convolutional Neural Networks (textCNN)
- Encoder-Decoder (seq2seq)
- Beam Search
- Attention Mechanism
- Machine Translation
- Appendix