CN109785903A - A kind of Classification of Gene Expression Data device - Google Patents

Abstract

The invention relates to the field of electrical data processing, in particular to a gene expression data classifier. Based on the traditional multi-task deep learning processing method, the present invention designs a gene expression data classifier with an input layer, a first hidden layer, a second hidden layer and an output layer. In particular, shared hidden units are set in the first hidden layer and the second hidden layer, so that the classifier can process gene expression data from different data sets and with different identifiers, which effectively solves the problem of classification of gene expression data. Tissue samples are insufficient and reduce the undesirable effects introduced by high feature space dimensionality.

Description

A Gene Expression Data Classifier

technical field

The invention relates to the field of electrical data processing, in particular to a gene expression data classifier.

Background technique

The study of the association between gene expression profiles and cancer/disease states is very important in biological and medical tasks. For example, comparing diseased and normal tissue can improve the understanding of pathology and help identify different tissues (cancerous or normal), as gene expression data can provide clues to tissue phenotype, function, and physiological processes. However, given the amount and complexity of gene expression data, traditional biological experiments cannot handle such data.

Table 1 Examples of gene expression data

Table 1 is an example of gene expression data. Gene expression data is typically represented using a matrix of n rows and m columns, where the rows represent traits (genes), and the columns represent samples (eg, tissue, disease stage, and treatment, etc.). Usually n >> m, it is unrealistic for a biological expert to manually calculate and compare the above n × m gene expression matrix. To this end, some machine learning algorithms are used to analyze gene expression and automatically classify them.

Over the past few decades, machine algorithms have been applied to microarrays in gene expression data and to classify human tissue into cancerous and normal tissue. The earliest application was decision tree (DT), which characterizes the sequence of proteins that have been diseased differently from all human proteins. Using K-nearestneighbor (K-NN) and naive Bayesian (naivbayesianClassifier, NB) to classify various types of gene expression data to identify human diseases, especially cancer. Along this direction, Bharathi used an Analysis of Variance (ANOVA) strategy for individual genes, while using a Support Vector Machine (SVM) to test the classification ability. Hu et al. proposed a Maximally Diversified Multiple Trees (MDMT) algorithm that aggregates a set of unique trees in decision factors. Halder et al. proposed a fuzzy k-nearest neighbor based active learning (ALFKNN) method, which first uses unlabeled samples to obtain labels from experts, and then iterates over labeled input samples. Added to training samples to improve prediction accuracy. Begum et al. combined the AdaBoost method with a linear support vector machine (AdaBoost and linear SVM, ADASVM) as a component classifier, and the results show that its performance is better than the current state-of-the-art classifiers.

However, existing classifiers face two challenges that make it difficult to directly apply to cancer diagnosis. The first challenge is the curse of dimensionality in the feature space in gene expression data. For example, the gene expression of thyroid cancer has 367 samples, and the number of features of each sample (the number of genes detected in this sample) is about the fourth power of the number of samples, which is a huge number. Therefore, the high dimensionality of the feature space increases the risk of overfitting because the number of genes is much larger than the number of samples. The second challenge is the lack of tissue samples, and it is difficult for existing methods to characterize cancer information well because there are few training data for cancer. There are two reasons for the insufficiency: 1) Rare cancers (some only occur in very few people each year) or rare traits are much more difficult to collect and obtain than data samples for common cancers or common traits. 2) It is difficult to integrate data samples from different gene expression platforms, which exacerbates the problem of insufficient data samples. For example, leukemia is one of the most common cancers in the world, causing 353,500 deaths in 2015. Many research groups study leukemia for the benefit of human health. However, due to different experimental settings, data sources from different research groups could not be integrated together to select for different gene expression signatures or even to label different cancer signatures. In the cancer dataset we tested, there were two types of leukemia data from two gene expression data sources. A leukemia dataset divides leukemia samples into two types (NPMI1+ and NPMI1-). Another leukemia dataset divides samples into four types (MP, HDMTX, HDMTX+MP, and LDMTX+MP). The first leukemia dataset was classified based on gene points, and the second was based on drug response of human leukemia cells. In addition, the first leukemia dataset has 54,675 gene signatures and the second leukemia dataset has 12,600 gene signatures, because different gene signatures are used to study the same cancer based on different background knowledge.

In the prior art, in order to solve the problem of insufficient number of tissue samples of gene expression data, there are commonly three classification methods of machine learning, namely: single-task learning, multi-task learning and transfer learning.

Figure 1 shows the differences in the learning process of single-task, multi-task and transfer learning techniques. In single-task learning (Fig. 1(a)), all 4 datasets are not connected, and each task will be trained separately, because single-task learning assumes that the training samples are independent of a specific distribution. Therefore, single-task learning completely ignores the relationship between related tasks. In multi-task learning (Fig. 1(b)), it assumes that tasks may be related, which means that information learned from one task can be used for another task. Hence, multi-task learning learns a joint model for all tasks simultaneously. In transfer learning (Fig. 1(c)), it transfers knowledge (parameters or expressions) from a source dataset to a target dataset to improve performance. In the prior art, multi-task learning and transfer learning have been fully demonstrated to significantly improve generalization performance when the number of samples is insufficient to train a single task.

As mentioned earlier, transfer learning extracts knowledge from the source dataset and transfers the knowledge to the target task, even though the training data and test data are applied to different domains, tasks and distributions. And transfer learning only pursues good performance on the target task because it cares more about the target dataset rather than the source dataset. In recent years, some transfer learning has achieved success in bioinformatics, especially biological image analysis. For example, Ravi.K.Samala et al. developed a computer-aided detection (CAD) system that can utilize deep convolution neural network (DCNN) to transfer knowledge from mammograms to Digital breast tomosynthesis (DBT). Hariharan et al. studied the transfer process of (convolution neural network, CNN), using the ImageNet dataset as the training set and medical images as the test set.

On the other hand, multi-task learning is close to transfer learning, which tries to learn multiple datasets (tasks) at the same time, even if all the datasets are different. But the goal of multi-task learning requires that all datasets have good experimental performance, not only the target task has good performance, because all datasets are very important for multi-task learning models. Multi-task learning is more suitable for the classification of gene expression data, and has achieved certain results in cancer data analysis. With the great success of deep learning techniques in image processing and pattern recognition, more and more researchers have combined multi-task learning and deep learning techniques into computer vision and bioinformatics over the past three years. In the field of computer vision, Zhang et al. proposed a task-constrained deep convolution network (TCDCN) model, which combines facial feature detection with multiple related tasks (such as head pose estimation task and facial attribute inference task). ) joint optimization. Similarly, Rejeev et al. proposed a CNN-based HyperFace architecture that can simultaneously perform face detection, facial feature localization, head pose evaluation, and gender recognition on a given image. Abrar et al. proposed a multi-task CNN model to better predict attributes in images, such as wearing a tie or a blue dress, by using a deep neural convolutional network (deep CNN). In the field of bioinformatics, Zhang et al. proposed a deep model based on transfer learning and multi-task learning for biological image analysis of domain-specific biological images. Ravi et al. proposed a multi-task transfer learning DCNN to transfer knowledge from non-medical images to a medical diagnosis learning task while learning auxiliary tasks. In the past five years, although the above-mentioned multi-task-based deep learning models have achieved success in computer vision and biomedical image analysis, the existing methods all complete the learning representation and obtain the learning results independently. Specifically, these methods learn parameters through a transfer learning approach and learn representations through a convolutional neural network (CNN) in the first stage. Then use multi-task learning techniques to produce each result of the task by learning a joint model from multiple datasets simultaneously.

SUMMARY OF THE INVENTION

In order to solve the problem of the scarcity of tissue samples and reduce the bad influence caused by the high dimension of the feature space, the present invention proposes a gene expression data classifier based on a multi-task deep learning algorithm (Multi-task Deep Learning, MTDL).

A gene expression data classifier proposed by the present invention is characterized in that, the classifier includes: an input layer, a first hidden layer, a second hidden layer and an output layer;

The input layer includes a plurality of input units, each input unit receives a data set, and the data unit is connected with a local hidden unit in the first shared unit;

The first hidden layer includes the same number of local hidden units as the number of input units in the input layer and a first shared hidden unit, the local hidden units are correspondingly connected to the local hidden units in the second hidden layer, and the first hidden unit is connected. Shared hidden units are connected to all input units;

The second hidden layer includes the same number of local hidden units as the input units in the input layer and a second shared hidden unit, the local hidden units are correspondingly connected to the output units in the output layer, and the second shared hidden unit Connect with all local hidden units and shared hidden units in the first hidden layer;

The output layer includes the same number of output units as the input units in the input layer, and each output unit is connected to the second hidden unit, where the output unit outputs the classification result;

The input unit, local hidden unit and shared hidden unit all output activation values.

Preferably, in the classifier, the first hidden layer and the second hidden layer can use a rectified linear unit (ReLU) function as the activation function of the local hidden unit, and the output layer uses a sigmoid as the activation function of the local hidden unit.

The gene expression data classifier proposed by the present invention has the advantages of both multi-task and deep learning. In particular, 1) the classifier can solve the problem of insufficient number of tissue samples and improve the negative impact of the high number of dimensions in the feature space; 2) even if the tumor types, characteristics, and even markers are different, the classifier can The source cancer database is integrated into one database to enhance the classification performance; 3) The classifier can continuously utilize multiple cancer databases to realize the discovery of hidden gene expression in small-scale cancer databases and enhance the classification performance.

Description of drawings

Figure 1 (a) single-task learning, (b) multi-task learning, (c) transfer learning;

Fig. 2 is the structure of the gene expression data classifier proposed by the present invention;

Fig. 3 is the classification accuracy comparison of the classifier proposed by the present invention, the DNN classifier, and the classifier using the sparse autoencoder;

Detailed ways

The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following examples are intended to illustrate the present invention, but not to limit the scope of the present invention.

The structure of a gene expression data classifier proposed by the present invention is shown in FIG. 2 .

A gene expression data classifier, wherein the classifier comprises: an input layer, a first hidden layer, a second hidden layer and an output layer;

The input layer includes a plurality of input units, each input unit receives a data set, and the data unit is connected with a local hidden unit in the first shared unit;

The first hidden layer includes the same number of local hidden units as the number of input units in the input layer and a first shared hidden unit, the local hidden units are correspondingly connected to the local hidden units in the second hidden layer, and the first hidden unit is connected. Shared hidden units are connected to all input units;

The shared hidden units are diamonds in Fig. 2, while triangles and squares represent multiple independent local hidden units in the first hidden layer and the second hidden layer, respectively.

The second hidden layer includes the same number of local hidden units as the input units in the input layer and a second shared hidden unit, the local hidden units are correspondingly connected to the output units in the output layer, and the second shared hidden unit Connect with all local hidden units and shared hidden units in the first hidden layer;

The output layer includes the same number of output units as the input units in the input layer, and each output unit is connected to the second hidden unit, where the output unit outputs the classification result;

The input unit, local hidden unit and shared hidden unit all output activation values.

Preferably, in the classifier, the first hidden layer and the second hidden layer can use a rectified linear unit (ReLU) function as the activation function of the local hidden unit, and the output layer uses a sigmoid as the activation function of the local hidden unit. In machine learning, non-saturating nonlinear ReLUs are much faster than saturating nonlinearities such as tanh and sigmoid when computing gradients. Therefore, a convolutional neural network (CNN) using ReLU units is trained several times faster than a convolutional neural network (CNN) using tanh units, and ReLU is chosen as the activation function for fast training. The sigmoid function is usually used to get the labels at the output layer because it only outputs 0 or 1. Therefore, the output of the sigmoid function is chosen to represent the classification results of multiple tasks.

Identify the inputs of n tasks using x ₁ , x ₂ , ..., x _n , for the first hidden layer, the activation values received by each local hidden unit It can be calculated by the following formula:

in, The superscript represents the index value of the layer, the subscript represents the index value of the task source, and the activation function is ReLU, for example, σ(x)=max(0,x); is the local weight of the edge between the No. 1 task source and the local hidden unit; is the bias value of the nth local hidden unit in the second hidden layer. In this way, the activation value s ¹ of the first shared hidden unit can be calculated by the following formula:

in, for The shared weight value between the boundary of s ¹ is the deviation of the shared hidden unit s ¹ , and the activation function used is ReLU. For the second hidden unit, each local hidden unit The activation value of can be calculated by:

in, for and Local weight value between boundaries; is the shared weight between the ith local hidden unit in the first hidden layer and the s2 boundary ^; is the bias of the ith local hidden unit in the second hidden layer. ^The activation value s2 of the shared hidden unit can be calculated by:

in, is the weight value between the ^s1 and ^s2 boundaries. For the output layer, the output of each task It can be calculated by the following formula:

in, for and local weight values between boundaries; for s ² with The weight value between the boundaries; is the deviation value of the output unit of the ith task. The activation function of the output unit is defined as:

sigmoid(x)=1/(1+e ^(-x) ).

The advantage of setting local hidden units and shared hidden units is that each task can learn private representations for classification from all local hidden units. Because local hidden units preserve the features of each individual task, different tasks are able to learn private representations of other tasks. On the other hand, shared hidden units learn the same representations from the entire dataset to fully exploit the information obtained from microarray systems. It is precisely because the shared hidden unit can utilize the information of all tasks, the shared hidden unit improves the performance of each task.

To sum up, the classifier can not only preserve the local properties of each task, but also utilize the shared knowledge to provide stable properties for all tasks.

To demonstrate the feasibility and effectiveness of this classifier, a total of 12 different datasets are used, as shown in Table 2. In Table 2, there are two acute myeloid leukemia datasets (task 1 and task 6) from different sources. It is impossible for traditional machine learning algorithms to integrate task 1 and task 6. To expand the acute myeloid leukemia dataset, use To address the technical problem of insufficient gene expression data. Because, A. Task 1 and Task 6 contain different labels in different domains. Among them, the "AML" and "MDS" tags in task 1 represent different stages of acute myeloid leukemia disease, but the "complete remission" and "relapse" tags in task 6 represent symptoms after different initial treatments; B . Task 1 and task 6 contain different types of gene features from different data sources, where 54,613 gene features were collected in task 1 and 12,625 gene features were collected in task 6. Because different research groups choose different labels and genetic signatures to investigate samples even for the same cancer, the problem of insufficient tissue samples in common cancer classifications remains unresolved.

Table 2 Summary of gene expression datasets

Further, the classifier using the MTDL algorithm (ie, the gene expression data classifier proposed by the present invention) and the classifier using two traditional deep learning methods (ie, deep neural network (DNN) and sparse autoencoder) are analyzed. Compared. Use leave-one-out cross-validation or evaluate DNN classifiers using 10 copies of the data. That is, 10 datasets are selected, of which 9 are used for training and 1 is used for testing. In order to eliminate the effect of random division, the experiment is repeated ten times, and the average precision of the obtained output results is used as the final result. The biggest difference between this classifier and the other two classifiers is that this classifier utilizes the shared knowledge of multiple datasets to learn more efficient representations, while the other two classifiers only learn local representations from datasets. In the experiment, the present invention proposes that the classifier can learn 12 datasets at the same time, but the other two classifiers can only learn the datasets independently.

Table 3. Accuracy of classifying 12 cancers using DNN classifier

Table 3 shows the classification accuracy of the traditional DNN classifier on the gene expression data of 12 tumor tissues. Since the traditional neural network model ignores the similarity information between datasets, the DNN classifier receives 12 cancer datasets and outputs the classification results one by one. Traditional DNN models ignore the similarity information between related cancer datasets, and if there are insufficient cancer samples, the classification performance will be quite poor.

Table 4. Accuracy of classification of 12 cancers using sparse autoencoder classifier

While DNN classifiers have very high accuracy on tasks 2, 3, 9, 10, 11, since there are only two labels in these cancer datasets, these samples are very easy to assign to each label. But the leukemia dataset (tasks 4, 5, 12) does not perform well in classification compared to other cancer datasets. There are two reasons: 1) due to the unclear pattern of each leukemia label, it is difficult to accurately diagnose leukemia; 2) there are more labels in task 5 (ie four labels) than other tasks, resulting in far lower accuracy. for other tasks.

The classifiers using sparse autoencoders in Table 4 have similar accuracy results for classifying cancer data information on 12 cancer datasets, because sparse autoencoders have not yet been able to use shared knowledge to improve classification performance.

Table 5 The accuracy of the classifier proposed by the present invention for classifying 12 cancers

Table 5 shows the classification accuracy of the proposed classifier for 12 cancers. Different from the contrasting classifiers, the classifier proposed in the present invention can process multiple cancer datasets (tasks) simultaneously, so that the classifier can make full use of the similar hidden information among all datasets (cancers) and improve each learning task. performance. From Table 5, it can be concluded that the accuracy of the leukemia dataset (tasks 4, 5 and 12) is greatly improved (over 20%) compared to the contrasting classifiers. Because the classifier proposed in the present invention can utilize these datasets simultaneously to reduce the ill effects of the lack of transparency problem in the leukemia dataset. In addition, the classifier proposed in the present invention can achieve more than 20% improvement in both task 7 and task 8, because other cancer datasets provide more representative information through shared layers to help task 7 and task 8 learn better , which improves the accuracy of classification results. Finally, the classifier proposed in the present invention has satisfactory performance on other cancer datasets, and some datasets themselves have clear classification patterns, making the classification results of the three classifiers all good. But the classifier proposed by the present invention still has the highest performance in most datasets, because the classifier proposed by the present invention is set up with shared hidden units, which can utilize all cancer datasets at the same time, and improve most of the data sets through the learning of shared hidden units. task classification performance.

In Figure 3, the accuracy results of all classifiers participating in the comparison are shown on 12 datasets, and the black line on the histogram in each subplot shows the standard deviation value of the accuracy for each task. It can be seen from FIG. 3 that the standard deviation of the classifier proposed by the present invention is much smaller than that of the comparison object, and not only has the best performance, but also has a more stable performance.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the technical principle of the present invention, several improvements and replacements can be made. These improvements and replacements It should also be regarded as the protection scope of the present invention.

Claims (6)

1. a gene expression data classifier, it is characterised in that the classifier comprises: an input layer, a first hidden layer, a second hidden layer and an output layer; The input layer includes a plurality of input units, each input unit receives a data set, and the data unit is connected with a local hidden unit in the first shared unit; The first hidden layer includes the same number of local hidden units as the number of input units in the input layer and a first shared hidden unit, the local hidden units are correspondingly connected to the local hidden units in the second hidden layer, and the first hidden unit is connected. Shared hidden units are connected to all input units; The second hidden layer includes the same number of local hidden units as the input units in the input layer and a second shared hidden unit, the local hidden units are correspondingly connected to the output units in the output layer, and the second shared hidden unit Connect with all local hidden units and shared hidden units in the first hidden layer; The output layer includes the same number of output units as the input units in the input layer, and each output unit is connected to the second hidden unit, where the output unit outputs the classification result; The input unit, local hidden unit and shared hidden unit all output activation values. 2. The classifier of claim 1, wherein the input unit uses the ReLU function to receive the source data set, and the activation value received by the local hidden unit in the first hidden It can be calculated by the following formula: in, The superscript of is the index value of the hidden layer, the subscript is the index value of the task source, and n is the number of input units in the input layer; is the local weight of the edge between the No. 1 task source and the local hidden unit; x _i is the input of the i-th task, is the bias value of the nth local hidden unit in the second hidden layer. 3. The classifier according to claim 2, wherein: the activation value s ¹ of the first shared hidden unit can be calculated by the following formula: in, for The shared weight value between the boundary of s ¹ is the deviation of the shared hidden unit s ¹ , and the activation function used is ReLU. 4. The classifier of claim 3, wherein the local hidden unit in the first hidden layer receives its activation value using the ReLU function, and the activation value received by each local hidden unit in the second hidden layer It can be calculated by the following formula: in, for and local weight values between boundaries, is the shared weight between the ith local hidden unit in the first hidden layer and the ^s2 boundary, is the bias of the ith local hidden unit in the second hidden layer. 5. The classifier according to claim 4, wherein: the activation value s of the ^second shared hidden unit can be calculated by the following formula: in, is the weight value between the ^s1 and ^s2 boundaries. 6. The classifier according to claim 4 or 5, wherein each output unit uses a sigmoid function to receive its activation value, and the output value of each task It can be calculated by the following formula: in, for and local weight values between boundaries, for s ² with the weight value between the boundaries, is the deviation value of the output unit of the ith task.