nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using TensorFlow and Pytorch. Most of the models in NLP were implemented with less than 100 lines of code.(except comments or blank lines)
- 1-1. NNLM(Neural Network Language Model) - Predict Next Word
- Paper - A Neural Probabilistic Language Model(2003)
- Colab - NNLM_Tensor.ipynb, NNLM_Torch.ipynb
- 1-2. Word2Vec(Skip-gram) - Embedding Words and Show Graph
- 1-3. FastText(Application Level) - Sentence Classification
- Paper - Bag of Tricks for Efficient Text Classification(2016)
- Colab - FastText.ipynb
- 2-1. TextCNN - Binary Sentiment Classification
- 2-2. DCNN(Dynamic Convolutional Neural Network)
- 3-1. TextRNN - Predict Next Step
- Paper - Finding Structure in Time(1990)
- Colab - TextRNN_Tensor.ipynb, TextRNN_Torch.ipynb
- 3-2. TextLSTM - Autocomplete
- Paper - LONG SHORT-TERM MEMORY(1997)
- Colab - TextLSTM_Tensor.ipynb, TextLSTM_Torch.ipynb
- 3-3. Bi-LSTM - Predict Next Word in Long Sentence
- Colab - Bi_LSTM_Tensor.ipynb, Bi_LSTM_Torch.ipynb
- 4-1. Seq2Seq - Change Word
- 4-2. Seq2Seq with Attention - Translate
- 4-3. Bi-LSTM with Attention - Binary Sentiment Classification
- 5-1. The Transformer - Translate
- 5-2. BERT - Classification Next Sentence & Predict Masked Tokens
| Model | Example | Framework | Lines(torch/tensor) |
|---|---|---|---|
| NNLM | Predict Next Word | Torch, Tensor | 67/83 |
| Word2Vec(Softmax) | Embedding Words and Show Graph | Torch, Tensor | 77/94 |
| TextCNN | Sentence Classification | Torch, Tensor | 94/99 |
| TextRNN | Predict Next Step | Torch, Tensor | 70/88 |
| TextLSTM | Autocomplete | Torch, Tensor | 73/78 |
| Bi-LSTM | Predict Next Word in Long Sentence | Torch, Tensor | 73/78 |
| Seq2Seq | Change Word | Torch, Tensor | 93/111 |
| Seq2Seq with Attention | Translate | Torch, Tensor | 108/118 |
| Bi-LSTM with Attention | Binary Sentiment Classification | Torch, Tensor | 92/104 |
| Transformer | Translate | Torch | 222/0 |
| Greedy Decoder Transformer | Translate | Torch | 246/0 |
| BERT | how to train | Torch | 242/0 |
- Python 3.5+
- Tensorflow 1.12.0+
- Pytorch 0.4.1+
- Plan to add Keras Version
- Tae Hwan Jung(Jeff Jung) @graykode
- Author Email : nlkey2022@gmail.com
- Acknowledgements to mojitok as NLP Research Internship.