@leriomaggio
|
valeriomaggio
|
valeriomaggio_at_gmail
|
git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git-
Part I: Introduction
-
Intro to Artificial Neural Networks
- Perceptron and MLP
- naive pure-Python implementation
- fast forward, sgd, backprop
-
Introduction to Deep Learning Frameworks
- Intro to Theano
- Intro to Tensorflow
- Intro to Keras
- Overview and main features
- Overview of the
corelayers - Multi-Layer Perceptron and Fully Connected
- Examples with
keras.models.SequentialandDense
- Examples with
- Keras Backend
-
-
Part II: Supervised Learning
-
Fully Connected Networks and Embeddings
- Intro to MNIST Dataset
- Hidden Leayer Representation and Embeddings
-
Convolutional Neural Networks
-
meaning of convolutional filters
- examples from ImageNet
-
Visualising ConvNets
-
Advanced CNN
- Dropout
- MaxPooling
- Batch Normalisation
-
HandsOn: MNIST Dataset
- FC and MNIST
- CNN and MNIST
-
Deep Convolutional Neural Networks with Keras (ref:
keras.applications)- VGG16
- VGG19
- ResNet50
-
-
Transfer Learning and FineTuning
-
Hyperparameters Optimisation
-
-
Part III: Unsupervised Learning
- AutoEncoders and Embeddings
- AutoEncoders and MNIST
- word2vec and doc2vec (gensim) with
keras.datasets - word2vec and CNN
- word2vec and doc2vec (gensim) with
-
Part IV: Recurrent Neural Networks
- Recurrent Neural Network in Keras
SimpleRNN,LSTM,GRU
- LSTM for Sentence Generation
- Recurrent Neural Network in Keras
-
PartV: Additional Materials:
- Custom Layers in Keras
- Multi modal Network Topologies with Keras
This tutorial requires the following packages:
-
Python version 3.5
- Python 3.4 should be fine as well
- likely Python 2.7 would be also fine, but who knows? :P
-
numpyversion 1.10 or later: http://www.numpy.org/ -
scipyversion 0.16 or later: http://www.scipy.org/ -
matplotlibversion 1.4 or later: http://matplotlib.org/ -
pandasversion 0.16 or later: http://pandas.pydata.org -
scikit-learnversion 0.15 or later: http://scikit-learn.org -
kerasversion 2.0 or later: http://keras.io -
tensorflowversion 1.0 or later: https://www.tensorflow.org -
ipython/jupyterversion 4.0 or later, with notebook support
(Optional but recommended):
pyyamlhdf5andh5py(required if you use model saving/loading functions in keras)- NVIDIA cuDNN if you have NVIDIA GPUs on your machines. https://developer.nvidia.com/rdp/cudnn-download
The easiest way to get (most) these is to use an all-in-one installer such as Anaconda from Continuum. These are available for multiple architectures.
I'm currently running this tutorial with Python 3 on Anaconda
!python --versionPython 3.5.2
In this repository, files to re-create virtual env with conda are provided for Linux and OSX systems,
namely deep-learning.yml and deep-learning-osx.yml, respectively.
To re-create the virtual environments (on Linux, for example):
conda env create -f deep-learning.ymlFor OSX, just change the filename, accordingly.
NOTE: Read this section only if after pip installing theano, it raises error in enabling the GPU support!
Since version 0.9 Theano introduced the libgpuarray in the stable release (it was previously only available in the development version).
The goal of libgpuarray is (from the documentation) make a common GPU ndarray (n dimensions array) that can be reused by all projects that is as future proof as possible, while keeping it easy to use for simple need/quick test.
Here are some useful tips (hopefully) I came up with to properly install and configure theano on (Ubuntu) Linux with GPU support:
- [If you're using Anaconda]
conda install theano pygpushould be just fine!
Sometimes it is suggested to install pygpu using the conda-forge channel:
conda install -c conda-forge pygpu
- [Works with both Anaconda Python or Official CPython]
-
Install
libgpuarrayfrom source: Step-by-step install libgpuarray user library -
Then, install
pygpufrom source: (in the same source folder)python setup.py build && python setup.py install -
pip install theano.
After Theano is installed:
echo "[global]
device = cuda
floatX = float32
[lib]
cnmem = 1.0" > ~/.theanorc
To date tensorflow comes in two different packages, namely tensorflow and tensorflow-gpu, whether you want to install
the framework with CPU-only or GPU support, respectively.
For this reason, tensorflow has not been included in the conda envs and has to be installed separately.
pip install tensorflowpip install tensorflow-gpuNote: NVIDIA Drivers and CuDNN must be installed and configured before hand. Please refer to the official Tensorflow documentation for further details.
All the code provided+ in this tutorial can run even if tensorflow is not installed, and so using theano as the (default) backend!
This is exactly the power of Keras!
Therefore, installing tensorflow is not stricly required!
+: Apart from the 1.2 Introduction to Tensorflow tutorial, of course.
By default, Keras is configured with theano as backend.
If you want to use tensorflow instead, these are the simple steps to follow:
- Create the
keras.json(if it does not exist):
touch $HOME/.keras/keras.json- Copy the following content into the file:
{
"epsilon": 1e-07,
"backend": "tensorflow",
"floatx": "float32",
"image_data_format": "channels_last"
}
- Verify it is properly configured:
!cat ~/.keras/keras.json{
"epsilon": 1e-07,
"backend": "tensorflow",
"floatx": "float32",
"image_data_format": "channels_last"
}
import numpy as np
import scipy as sp
import pandas as pd
import matplotlib.pyplot as plt
import sklearnimport kerasUsing TensorFlow backend.
import numpy
print('numpy:', numpy.__version__)
import scipy
print('scipy:', scipy.__version__)
import matplotlib
print('matplotlib:', matplotlib.__version__)
import IPython
print('iPython:', IPython.__version__)
import sklearn
print('scikit-learn:', sklearn.__version__)numpy: 1.11.1
scipy: 0.18.0
matplotlib: 1.5.2
iPython: 5.1.0
scikit-learn: 0.18
import keras
print('keras: ', keras.__version__)
# optional
import theano
print('Theano: ', theano.__version__)
import tensorflow as tf
print('Tensorflow: ', tf.__version__)keras: 2.0.2
Theano: 0.9.0
Tensorflow: 1.0.1