Dive into Deep Learning - booknotes

(d2l.ai)
intro
example
data, models, algorithms
regression
classification
tagging
search & rank
recommenders
sequence learning
reinforcement learning
markov decision processes
preliminaries
data manipulation
preprocessing
linear algebra
calculus
auto differentiation
probability
documentation
linear neural nets
linear regression
LR from scratch
LR implementation
softmax regression
fashion-mnist
softmax from scratch
softmax implementation
multilayer perceptrons
intro
MLP from scratch
model selection, underfit, overfit
weight decay
dropout
fwd/back propagation
computational graphs
numerical stability
environmental concerns
kaggle - house prices
computation
layers & blocks
parameters
deferred initialization
custom layers
file i/o
gpus
CNNs
dense layers to convolutions
image convolutions
padding, stride
channels
pooling
LeNet
Modern CNNs
alexnet
vgg
network-in-network (NiN)
googlenet
batch normalization
resnet
densenet
RNNs
sequence models
text preprocessing
language models
rnns
rnns from scratch
implementation
backprop through time
Modern RNNs
gated recurrent unit (grus)
lstms
deep rnns
bidirectional rnns
machine translation
encoder-decoders
sequence-to-sequence
beam search
Attention Mechanisms
intro
sequence-to-sequence
transformers
Optimization
intro
convexity
gradient descent
stochastic gradient descent
minibatch SGD
momentum
adagrad
rmsprop
adadelta
adam
learning rate scheduling
Performance
compilers & interpreters
async computation
auto parallelism
hardware
multiple GPUs
parameter servers
Vision
image augmentation
fine tuning
object detection / bounding boxes
anchor boxes
multiscale OD
pikachu dataset
single-shot multibox detect
region-based CNNs (R-CNNs)
semantic segmentation
transposed convolution
fully convolutional nets (FCNs)
neural style transfer
CIFAR-10 image class on Kaggle
Imagenet/Dogs on Kaggle
NLP
word2vec
word2vec - approx training
word2vec dataset
word2vec implementation
subword embedding (fasttext)
word embed - global vectors (GloVe)
synonyms & analogies
text sentiment classification
text sentiment using RNNs
text sentiment using textCNN
Recommenders
overview
MovieLens
matrix factorization
AutoRec with autoencoders
personalized rankings
collaborative filtering
sequence-aware recommenders
feature-rich recommenders
factorization machines
deep factorization machines
GANs
intro
deep convolutional GANs
Appendix: Math
geometry
linear algebra
eigendecomposition
single-variable calculus
multi-variable calculus
integral calculus
random variables
max likelihood
naive bayes
statistics
info theory
Appendix: Tools
jupyter
amazon sagemaker
aws ec2
google colab
servers & gpus