PyTorch Tutorials
0.2.0_4
Beginner Tutorials
Deep Learning with PyTorch: A 60 Minute Blitz
What is PyTorch?
Getting Started
Tensors
Operations
Numpy Bridge
Converting torch Tensor to numpy Array
Converting numpy Array to torch Tensor
CUDA Tensors
Autograd: automatic differentiation
Variable
Gradients
Neural Networks
Define the network
Loss Function
Backprop
Update the weights
Training a classifier
What about data?
Training an image classifier
1. Loading and normalizing CIFAR10
2. Define a Convolution Neural Network
3. Define a Loss function and optimizer
4. Train the network
5. Test the network on the test data
Training on GPU
Where do I go next?
PyTorch for former Torch users
Tensors
Inplace / Out-of-place
Zero Indexing
No camel casing
Numpy Bridge
Converting torch Tensor to numpy Array
Converting numpy Array to torch Tensor
CUDA Tensors
Autograd
Variable
Gradients
nn package
Example 1: ConvNet
Forward and Backward Function Hooks
Example 2: Recurrent Net
Multi-GPU examples
DataParallel
Part of the model on CPU and part on the GPU
Learning PyTorch with Examples
Tensors
Warm-up: numpy
PyTorch: Tensors
Autograd
PyTorch: Variables and autograd
PyTorch: Defining new autograd functions
TensorFlow: Static Graphs
nn
module
PyTorch: nn
PyTorch: optim
PyTorch: Custom nn Modules
PyTorch: Control Flow + Weight Sharing
Examples
Tensors
Warm-up: numpy
PyTorch: Tensors
Autograd
PyTorch: Variables and autograd
PyTorch: Defining new autograd functions
TensorFlow: Static Graphs
nn
module
PyTorch: nn
PyTorch: optim
PyTorch: Custom nn Modules
PyTorch: Control Flow + Weight Sharing
Transfer Learning tutorial
Load Data
Visualize a few images
Training the model
Visualizing the model predictions
Finetuning the convnet
Train and evaluate
ConvNet as fixed feature extractor
Train and evaluate
Data Loading and Processing Tutorial
Dataset class
Transforms
Compose transforms
Iterating through the dataset
Afterword: torchvision
Deep Learning for NLP with Pytorch
Introduction to PyTorch
Introduction to Torch’s tensor library
Creating Tensors
Operations with Tensors
Reshaping Tensors
Computation Graphs and Automatic Differentiation
Deep Learning with PyTorch
Deep Learning Building Blocks: Affine maps, non-linearities and objectives
Affine Maps
Non-Linearities
Softmax and Probabilities
Objective Functions
Optimization and Training
Creating Network Components in Pytorch
Example: Logistic Regression Bag-of-Words classifier
Word Embeddings: Encoding Lexical Semantics
Getting Dense Word Embeddings
Word Embeddings in Pytorch
An Example: N-Gram Language Modeling
Exercise: Computing Word Embeddings: Continuous Bag-of-Words
Sequence Models and Long-Short Term Memory Networks
LSTM’s in Pytorch
Example: An LSTM for Part-of-Speech Tagging
Exercise: Augmenting the LSTM part-of-speech tagger with character-level features
Advanced: Making Dynamic Decisions and the Bi-LSTM CRF
Dynamic versus Static Deep Learning Toolkits
Bi-LSTM Conditional Random Field Discussion
Implementation Notes
Exercise: A new loss function for discriminative tagging
Intermediate Tutorials
Classifying Names with a Character-Level RNN
Preparing the Data
Turning Names into Tensors
Creating the Network
Training
Preparing for Training
Training the Network
Plotting the Results
Evaluating the Results
Running on User Input
Exercises
Generating Names with a Character-Level RNN
Preparing the Data
Creating the Network
Training
Preparing for Training
Training the Network
Plotting the Losses
Sampling the Network
Exercises
Translation with a Sequence to Sequence Network and Attention
Loading data files
The Seq2Seq Model
The Encoder
The Decoder
Simple Decoder
Attention Decoder
Training
Preparing Training Data
Training the Model
Plotting results
Evaluation
Training and Evaluating
Visualizing Attention
Exercises
Reinforcement Learning (DQN) tutorial
Replay Memory
DQN algorithm
Q-network
Input extraction
Training
Hyperparameters and utilities
Training loop
Writing Distributed Applications with PyTorch
Setup
Point-to-Point Communication
Collective Communication
Distributed Training
Our Own Ring-Allreduce
Advanced Topics
Communication Backends
Initialization Methods
Advanced Tutorials
Neural Transfer with PyTorch
Introduction
Neural what?
How does it work?
OK. How does it work?
PyTorch implementation
Packages
Cuda
Load images
Display images
Content loss
Style loss
Load the neural network
Input image
Gradient descent
Creating extensions using numpy and scipy
Parameter-less example
Parametrized example
Transfering a model from PyTorch to Caffe2 and Mobile using ONNX
Transfering SRResNet using ONNX
Running the model on mobile devices
Custom C extensions for pytorch
Step 1. prepare your C code
Step 2: Include it in your Python code
PyTorch Tutorials
Docs
»
Index
Index