TextSentimentClassification

TextSentimentClassification, using tensorflow. Original Data

Data Preprocessing

Remove the letter whose number of repetitions is over 3 from a word...

Word Vectors Training

Using word2vec and GloVe to generate word vectors...

Models

Performance

Model	Epoch	Training Accuracy	Validation Accuracy	Parameters(word vectors excluded)
TextCNN+nonstatic	130	0.8839	0.8142	281,202
TextRNN+nonstatic	150	0.8383	0.8199	285,826
CRNN+nonstatic	70	0.8600	0.8219	274,818
RCNN+nonstatic	50	0.8553	0.8227	318,978
HAN+nonstatic	110	0.8355	0.8188	209,410

TextCNN

Reference

Convolutional Neural Networks for Sentence Classification

A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification

Model Architecture

Total 4 ways:

• CNN-rand
• CNN-static
• CNN-nonstatic
• CNN-multichannel

Performance

Model	Epoch	Training Accuracy	Validation Accuracy	Parameters(word vectors excluded)
TextCNN+rand	130	0.8761	0.8137	281,202
TextCNN+static	60	0.9015	0.8113	281,202
TextCNN+nonstatic	130	0.8839	0.8142	281,202
TextCNN+multichannel	60	0.9225	0.8141	561,202

Choosing to use word vectors in a nonstatic way.

TextRNN

Model Architecture

Using bidirectional RNN, and then concatenating the output of the forward process and the output of the backward process...

CRNN

Reference

A C-LSTM Neural Network for Text Classification

Model Architecture

Using CNN to extract sentences with higher-level phrase representations, and then learning long short-term dependency with bi-RNN...

RCNN

Reference

Recurrent Convolutional Neural Networks for Text Classification

Model Architecture

In addition to implementing the same structure as the paper, using bi-LSTM or bi-GRU and then concatenating their outputs... RNN for capturing contextual information and max pooling used for judging which words play key roles in the task...

HAN

Reference

Hierarchical Attention Networks for Document Classification

Model Architecture

Transforming a sentence into a document consisting of sentences...

Ensembles

Bagging

Uniform blending...

Stacking

Using Logistic Regression as the level-2 classifier...

Performance

Model	Epoch	Training Accuracy	Testing Accuracy	Parameters(word vectors excluded)
LR+static_avg	-	0.77364	0.773605	-
NB+static_avg	-	0.606435	0.61082	-
TextCNN+nonstatic	130	0.8703	0.817615	281,202
TextRNN+nonstatic	150	0.8384	0.81969	285,826
CRNN+nonstatic	70	0.8589	0.82449	274,818
RCNN+nonstatic	50	0.8497	0.822935	318,978
HAN+nonstatic	110	0.8330	0.820235	209,410
bagging	-	0.8538	0.82999	-
stacking	-	0.867135	0.831045	-