CN112735514B - A training and visualization method and system for neural network extraction and regulation of DNA combination patterns - Google Patents

Abstract

The invention discloses a training and visualization method and system for a neural network to extract and regulate a DNA combination pattern. The method includes: obtaining a DNA sequence with a specific function and a DNA sequence without a specific function; labeling the two DNA sequences, and Use one-hot encoding to represent DNA sequences with specific functions and DNA sequences without specific functions; build a convolutional neural network, take the one-hot encoding of the labeled DNA sequence as input, and label the corresponding DNA sequence as the output of the convolutional neural network The fitted value of the convolutional neural network is trained to make the convolutional neural network recognize DNA sequences; the trained convolutional neural network is decoupled using the NeuronMotif algorithm to obtain the gene regulatory element combination module, and the regulatory element syntax tree is used. representation and storage. This method provides a general neural network interpretation algorithm NeuronMotif that decouples convolutional neural networks to discover and visualize patterns recognized by convolutional neural networks.

Description

A training and visualization method and system for neural network extraction and regulation of DNA combination patterns

technical field

The invention relates to the technical field of gene regulation, in particular to a method and system for training and visualization of a neural network extracting and regulating DNA combination mode.

Background technique

Gene expression and regulation determine the growth and differentiation of cells, and the transcriptional regulation process of controlling genes can control the level of gene expression to a certain extent and then control various states of cells. In the process of gene transcription regulation, the combinational arrangement logic of various regulatory elements on genomic DNA is one of the most critical factors. In the application of gene editing and transformation, it can be designed and adjusted according to the needs of specific gene functions, according to the logic of base preference, distance position, sequence, and number of occurrences of multiple regulatory elements, so as to achieve the control of gene transcription level. . However, such complex control modules and logic are difficult to extract and represent with current shallow machine learning methods and models. Deep learning models perform well in many genome functional annotation tasks due to their complex performance capabilities and excellent automatic feature extraction capabilities, but the learned combination modules of gene regulatory elements are difficult to interpret and extract.

In recent years, a lot of work has been done to study the extraction method of gene regulatory element combination modules in neural networks, and some progress has been made, but these advances have not solved the problem. At present, the idea of interpreting neural networks in the problem of DNA sequence prediction is basically the same, which is to study the relationship between the bases input by neurons and the output of neurons. The method is basically improved from the field of computer vision, and can also be applied to Visualization of neural networks in computer vision or other fields. These methods can basically be divided into three categories: (1) changing the input to see the change in the output value; (2) the back gradient propagation algorithm; (3) the sequence input distribution that maximizes the activation value. They explain the neural network to a certain level, but they all ignore that the neural network is a hybrid model, and there is no way to try to open the neural network black box to solve this problem.

The typical representative of this method of changing the input to see the change of the output value is DeepSEA. The advantage of this method is that it is the most simple and straightforward, making it easy to understand. If the input base has changed, but the output neuron has not changed, the base is not a critical base, otherwise it means that the base is very important. The main disadvantage of this method is that it is extremely computationally intensive, and the number of combinations that change at each base position grows exponentially with the length of the DNA sequence. This method is mostly suitable for the study of single nucleotide polymorphisms. It is concerned with the effect of mutations at a few sites on the function of a sequence, rather than studying all base positions, so it can basically meet the needs of users. need. This kind of analysis does not seem to show the whole picture of the knowledge learned by the neural network. Most of the work on neural network analysis is not limited to this method, so its application is not particularly extensive.

For the other two methods, they borrow from methods commonly used in the image field in recent years and can be used to resolve the importance of all bases in each sample. The two methods are implemented using the back gradient propagation algorithm, but the specific methods are different. Saliency Map and DeepLIFT are typical representatives of analytical methods based on the back gradient propagation algorithm, which use the partial derivative of the neuron output value to the input value or similar deformation as the importance evaluation of the input position. This method can be easily solved using the back gradient propagation algorithm, so it can be easily applied to any neuron. The user only needs to provide a sequence to be studied, input it into the neural network, propagate forward once, and then calculate some The gradient is back-propagated once to complete the importance annotation of the corresponding position in the sequence. Because of its lower computational cost, it is relatively more widely used, but it also has quite a few problems. One of the problems is that it cannot directly calculate Motif. Motif is the probability distribution statistics for all base positions of multiple sequences, and this method only provides an evaluation of the importance of the corresponding position of a sequence, so it has no statistical significance. In order to meet this demand, the research group based on the DeepLIFT algorithm has developed TF-MoDISco. Its basic idea is to perform a series of post-processing such as matching, aligning, cutting, and clustering on the key subsequences in some sequences of interest, and finally The importance scores for each base position of multiple sequences are combined. But the problem is that the importance scores of the corresponding positions of each sequence are not comparable, the relative size has no absolute significance, and the calculation operation process is very dependent on manual settings, and the results are not particularly stable, so the calculated or found The so-called "Motif" has not been widely used.

The sequence input distribution that maximizes the activation value is mainly due to the characteristics of the neural network itself. Any neuron can affect the function of the next layer of neurons only when it is activated, which means that the sequence it recognizes is the sequence that enables it to be activated. Calculate PPM (PositionProbabilityMatrix) and PWM (PositionWeightMatrix). But there are also a lot of problems, such as how the thresholds of these sequences should be selected, and no one has given a reasonable explanation. In the example of explaining the Basset model, the author uses the corresponding volume to explain the Motif learned by the neurons in the first layer. The accumulation kernel scans all samples, selects the sequence whose activation value is more than half of the obtained maximum value as the activated sequence set, uses this set to calculate the PWM and draws the WebLogo diagram corresponding to the Motif. These Motifs are similar to the Motif in the standard database. Satisfactory, but the reason why the threshold is half of the maximum value is not explained, and other works have similar problems. While the Basset model has yielded good results on the interpretation of neurons in the first layer, so far there has been little work using this method to reasonably parse what motifs are learned by neurons in the second layer and above. This suggests that this approach may no longer be directly applicable at the second or deeper level.

In view of the above three aspects, the current method for extracting gene regulatory element combination modules in neural networks has encountered a bottleneck, and a better method is needed to extract the learned gene regulatory element combination modules in neural networks.

SUMMARY OF THE INVENTION

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

To this end, an object of the present invention is to propose a training and visualization method for extracting and regulating DNA combination patterns using a neural network, which provides a general interpretation algorithm NeuronMotif for decoupling neural networks. NeuronMotif can be decoupled to annotate whether DNA is A convolutional neural network model with specific functions to discover and visualize the combined modules of gene regulatory elements identified in it. The algorithm can also be used to discover and visualize patterns identified by any convolutional neural network in other problems or domain applications.

Another object of the present invention is to propose a training and visualization system for extracting and regulating DNA combination patterns using neural networks.

In order to achieve the above object, an embodiment of the present invention proposes a training and visualization method for extracting and regulating DNA combination patterns by a neural network, including:

S1, obtaining DNA sequences with specific functions and DNA sequences without the specific functions;

S2, annotating the two DNA sequences, and expressing the DNA sequence with a specific function and the DNA sequence without the specific function using one-hot encoding;

S3, build a convolutional neural network, take the one-hot encoding of the marked DNA sequence as input, and mark the corresponding DNA sequence as the fitting value of the output of the convolutional neural network, and train the convolutional neural network to make the convolutional neural network identify DNA sequences;

S4, the trained convolutional neural network is decoupled by NeuronMotif algorithm to obtain a combination module of gene regulatory elements, which is represented and stored by the syntax tree of regulatory elements.

The training and visualization method of the neural network extraction and regulation DNA combination pattern according to the embodiment of the present invention, by obtaining DNA sequences with specific functions and DNA sequences without specific functions; Sequences and DNA sequences without specific functions are represented by one-hot encoding; a convolutional neural network is built, and the one-hot encoding of the labeled DNA sequence is used as input, and the corresponding DNA sequence is labeled as the fitting value of the output of the convolutional neural network. The convolutional neural network is trained to make the convolutional neural network recognize DNA sequences; the NeuronMotif algorithm is designed and used to decouple the trained convolutional neural network, so as to discover the Motif and Motif combination modules corresponding to each neuron, and obtain the gene The regulatory element combination module is represented and stored using the regulatory element syntax tree, which provides a new set of ideas and solutions for the extraction of the gene regulatory element combination module in the neural network.

In addition, the training and visualization method for extracting and regulating DNA combination patterns by the neural network according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Further, in an embodiment of the present invention, S1 further includes:

S11, cutting out the DNA sequence fragments with the specific function and the DNA sequence fragments without the specific function from the biological genome marked by the biological experiment method.

Further, in an embodiment of the present invention, S1 further includes:

S12, by artificially synthesizing DNA sequence fragment molecules, any type of biological function verification experiment is performed to determine the fragment molecules that have the specific function and the fragment molecules that do not have the specific function.

Further, in an embodiment of the present invention, the annotating the DNA sequence includes:

The DNA sequence with the specific function is marked as a positive sample, and the DNA sequence without the specific function is marked as a negative sample.

Further, in an embodiment of the present invention, S4 further comprises:

S41, for a neuron in the convolutional neural network, collect a new set of DNA sequences, where different DNA sequences in the new set of DNA sequences have neuron activation values of various sizes;

S42, respectively calculating all the neuron activation values that the DNA sequences in the new set of DNA sequences can affect the neuron at each layer of the neural network;

S43, dividing the new set of DNA sequences to obtain a plurality of subsets of DNA sequences;

S44, calculating the mathematical representation of the gene function element combination module corresponding to each DNA sequence subset, and using the regulatory element syntax tree to represent and store the gene function element combination module.

Further, in an embodiment of the present invention, S41 further includes:

The DNA sequence is randomly generated according to the size of the neuron receptive field, and the DNA sequence is optimized by using the genetic algorithm. The optimization goal is the neuron activation value of the DNA sequence. The mutation of the DNA sequence in the genetic algorithm is based on the neuron activation value to the DNA sequence. The gradient size of the one-hot encoding input is sampled as a probability. In addition to maintaining the cross-exchange of DNA sequences, it is also necessary to perform cyclic displacement according to the neural network pooling layer structure to sample the intermediate result DNA sequence optimized by the genetic algorithm. The sampled DNA The sequences are not repeated, and the sampled DNA sequences form a collection of various activated DNA sequences.

Further, in one embodiment of the present invention, S43 further comprises:

S431, for the new set of DNA sequences, start from the layer where the neuron is located, and perform detection from deep to shallow layers. If a maximum pooling layer is encountered, use the Kmeans algorithm according to the DNA sequence according to the pooling size K. The sequence of the new set corresponds to the neuron activation value feature of the shallow layer of the pooling layer, and the new set of DNA sequences is aggregated into K categories, each category corresponding to the divided DNA sequence subsets;

S432, the divided DNA sequence subsets are regarded as a new DNA sequence set, starting from the clustering layer, and then detecting from the deep layer to the shallow layer. If the maximum pooling layer is encountered, Kmeans is used according to the pooling size K. The algorithm aggregates the new set of DNA sequences into K categories according to the activation value characteristics of the neurons in the shallower layer of the pooling layer corresponding to the sequences of the new set of DNA sequences, each of which corresponds to the divided DNA sequence subsets;

S433, repeating step S432 until the first layer, to obtain the plurality of DNA sequence subsets.

Further, in an embodiment of the present invention, the calculation expression of the gene function element combination module is E[E(X|Y)], wherein X is the random variable corresponding to the one-hot encoding of the sampling sequence, Y is a random variable represented by the activation value corresponding to the sampling sequence, and the relationship between Y and X is determined by the corresponding neuron Y=f(X), where the distribution of the random variable Y needs to be given, which is a free variable, a random variable X depends on random variable Y.

In order to achieve the above object, another embodiment of the present invention proposes a training and visualization system for a neural network to extract and regulate DNA combination patterns, including:

an acquisition module for acquiring DNA sequences with specific functions and DNA sequences without the specific functions;

An annotation module, configured to annotate two DNA sequences, and express the DNA sequence with a specific function and the DNA sequence without the specific function using one-hot encoding;

The training module is used to build a convolutional neural network. The one-hot encoding of the marked DNA sequence is used as input, and the corresponding DNA sequence is marked as the fitting value of the output of the convolutional neural network. The integrated neural network recognizes DNA sequences;

The decoupling module is used to decouple the trained convolutional neural network through the NeuronMotif algorithm to obtain the gene regulatory element combination module, which is represented and stored by the regulatory element syntax tree.

The training and visualization system of the neural network extraction and regulation DNA combination pattern of the embodiment of the present invention obtains DNA sequences with specific functions and DNA sequences without specific functions; annotates the two DNA sequences, and combines the DNA sequences with specific functions Sequences and DNA sequences without specific functions are represented by one-hot encoding; a convolutional neural network is built, and the one-hot encoding of the labeled DNA sequence is used as input, and the corresponding DNA sequence is labeled as the fitting value of the output of the convolutional neural network. The convolutional neural network is trained to make the convolutional neural network recognize DNA sequences; the NeuronMotif algorithm is designed and used to decouple the trained convolutional neural network, so as to discover the Motif and Motif combination modules corresponding to each neuron, and obtain the gene The regulatory element combination module is represented and stored using the regulatory element syntax tree, which provides a new set of ideas and solutions for the extraction of the gene regulatory element combination module in the neural network.

In addition, the training and visualization system for the neural network extraction and regulation DNA combination pattern according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Further, in one embodiment of the present invention, obtaining a DNA sequence with a specific function and a DNA sequence without the specific function, including:

Cut out DNA sequence fragments with the specific function and DNA sequence fragments without the specific function from the biological genome annotated by biological experimental means; or

By artificially synthesizing DNA sequence fragment molecules, any type of biological function verification experiment is performed to determine the fragment molecules that have the specific function and the fragment molecules that do not have the specific function.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a flowchart of a training and visualization method for extracting and regulating DNA combination patterns by a neural network according to an embodiment of the present invention;

2 is a schematic diagram of a PPM in mathematical form according to an embodiment of the present invention;

3 is a schematic diagram of transcription factor matching according to an embodiment of the present invention;

4 is a schematic diagram of a syntax tree according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a training and visualization system for extracting and regulating DNA combination patterns by a neural network according to an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

The following describes the training and visualization method and system for extracting and regulating DNA combination patterns by neural network according to the embodiments of the present invention with reference to the accompanying drawings.

First, a method for training and visualizing a neural network extraction and regulation DNA combination pattern proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a flowchart of a training and visualization method for extracting and regulating DNA combination patterns by a neural network according to an embodiment of the present invention.

As shown in Figure 1, the training and visualization method of the neural network for extracting and regulating DNA combination patterns includes the following steps:

Step S1, obtaining DNA sequences with specific functions and DNA sequences without specific functions.

Further, in the embodiments of the present invention, two methods for collecting DNA sequences are provided. The first is to cut functional DNA sequence fragments and non-functional DNA sequence fragments from biological genomes annotated by various biological experimental methods, such as DNA sequences of chromatin open regions and chromatin non-open regions annotated by ATAC-seq DNA sequences of regions, DNA sequences of nucleosomal modifications or transcription factor binding sites annotated by ChIP-seq and DNA sequences without nucleosome modifications or transcription factor binding sites.

The second is to artificially synthesize DNA sequence fragments, perform any type of biological function verification experiments, and determine the functional fragments and non-functional fragments. Sequences that bind to proteins and sequences that do not bind to proteins.

In step S2, the two DNA sequences are annotated, and the DNA sequences with specific functions and the DNA sequences without specific functions are represented by one-hot encoding.

Further, annotate the DNA sequence, including:

The fixed-length DNA sequences with specific functions are marked as positive samples, and the value is marked as 1; the DNA sequences without specific functions are marked as negative samples, and the value is marked as 0. For each DNA sequence, multiple functions are allowed, so multiple annotations can appear, corresponding to whether they have the corresponding function or not.

In step S3, a convolutional neural network is built, the one-hot encoding of the marked DNA sequence is used as input, the corresponding DNA sequence is marked as the fitting value of the output of the convolutional neural network, and the convolutional neural network is trained to make the convolutional neural network. The network recognizes DNA sequences.

Specifically, DNA sequences with functions are marked as positive samples, DNA sequences without functions are marked as negative samples, and DNA sequences are represented by one-hot encoding. Build any convolutional neural network. The structure can include convolutional layers, pooling layers, fully connected layers, etc. The input dimension should match the length of the DNA sequence and the one-hot encoding format, and the output dimension should match the number of DNA functions, using the one-hot DNA sequence. The code is used as input, and the corresponding DNA sequence is marked as the fitting value of the neural network output, and the neural network is trained so that the neural network can identify whether the DNA sequence is a positive sample as accurately as possible.

In step S4, the trained convolutional neural network is decoupled through the NeuronMotif algorithm to obtain a gene regulatory element combination module, which is represented and stored by using the regulatory element syntax tree.

The NeuronMotif algorithm is a general algorithm for decoupling convolutional neural networks that decouple convolutional neural network models used to annotate whether DNA has a specific function, discover and visualize the combined modules of gene regulatory elements identified therein , the algorithm can also be used to discover and visualize patterns identified by convolutional neural networks in other problem applications.

Further, S4 further includes:

S41, for a neuron in the convolutional neural network, collect a new set of DNA sequences, where different DNA sequences in the new set of DNA sequences have neuron activation values of various sizes;

S42, respectively calculating all the neuron activation values that the DNA sequences in the new set of DNA sequences can affect the neuron at each layer of the neural network;

S43, dividing the new set of DNA sequences to obtain a plurality of subsets of DNA sequences;

S44, calculating the mathematical representation of the gene function element combination module corresponding to each DNA sequence subset, and using the regulatory element syntax tree to represent and store the gene function element combination module.

It is understandable that for each neuron in the convolutional neural network, the process of S41-S44 needs to be completed from the shallow layer to the deep layer.

Further, in an embodiment of the present invention, S41 further includes:

The DNA sequence is randomly generated according to the size of the neuron receptive field, and the DNA sequence is optimized by using the genetic algorithm. The optimization goal is the neuron activation value of the DNA sequence. The mutation of the DNA sequence in the genetic algorithm is based on the neuron activation value to the DNA sequence. The gradient size of the one-hot encoding input is sampled as a probability. In addition to maintaining the cross-exchange of DNA sequences, it is also necessary to perform cyclic displacement according to the neural network pooling layer structure to sample the intermediate result DNA sequence optimized by the genetic algorithm. The sampled DNA The sequences are not repeated, and the sampled DNA sequences form a collection of various activated DNA sequences.

It is understandable that if the number of sampled DNA sequences is too large, it will be divided into 20 or more activation value intervals according to the maximum activation value, and the DNA sequences in each interval will be selected randomly and non-repeatedly, and those that have not been selected will be discarded. DNA sequence samples.

Further, in one embodiment of the present invention, S43 further comprises:

S431, for the new set of DNA sequences, start from the layer where the neuron is located, and perform detection from the deep layer to the shallow layer. If the maximum pooling layer is encountered, according to the pooling size K, use the Kmeans algorithm according to the sequence corresponding to the new set of DNA sequences. The feature of the activation value of the shallow neurons in the pooling layer aggregates a new set of DNA sequences into K categories, each of which corresponds to a subset of the divided DNA sequences;

S432, the divided DNA sequence subsets are regarded as a new DNA sequence set, starting from the clustering layer, and then detecting from the deep layer to the shallow layer. If the maximum pooling layer is encountered, Kmeans is used according to the pooling size K. The algorithm aggregates the new set of DNA sequences into K categories according to the activation value characteristics of the neurons in the shallower layer of the pooling layer corresponding to the sequences of the new set of DNA sequences, each of which corresponds to the divided DNA sequence subsets;

S433, repeating step S432 until the first layer, to obtain a plurality of DNA sequence subsets.

Further, in an embodiment of the present invention, the calculation expression of the gene function element combination module is E[E(X|Y)], wherein X is the random variable corresponding to the one-hot encoding of the sampling sequence, Y is a random variable represented by the activation value corresponding to the sampling sequence, and the relationship between Y and X, Y=f(X), is determined by the corresponding neuron, where the distribution of the random variable Y needs to be given, which is a free variable, while random The variable X depends on the random variable Y. Here, the probability density function corresponding to the distribution of Y is recommended to be p(y)=2y/(A*A), where A is the maximum activation value of all sequences in the DNA sequence set. E[E(X|Y)] represents the expected value of the one-hot encoding of the DNA sequence samples taken under the condition of a given activation value distribution. E[E(X|Y)] is calculated for each sub-set of divided DNA sequence samples, and the combination module of gene function elements represented by the neuron can be obtained.

For all the obtained gene functional element combination modules, find the same set of functional element patterns represented as {A,B,C…}, determine the length, arrangement and relative position of the functional elements in each combination module, in general, in In the combination module, their arrangement is fixed, for example, they are all ABCABC, and the number of bases spaced between two adjacent elements is inconsistent. If in all combination modules, the distance between adjacent elements is a fixed number of bases, insert the number of bases between two adjacent elements in the arrangement, such as A-6N-B, where 6N represents a distance of 6 bases in length , and for the uncertain distance, the interval can be written and divided by brackets, such as A-[6±2N]-B. Sub-modules with relatively fixed or related functions can be marked with brackets, and modules can be nested in modules, such as: [A-6N-[B-C]]-[A-6N-[B-C]], which can be expressed according to brackets Get the syntax tree of the function module.

Specifically, the method of decoupling any one convolutional neuron in the neural network is described:

1) For a neuron, collect a new set of DNA sequences, which has a sufficient number of new sets of DNA sequences of various activation values.

The DNA sequence is randomly generated according to the size of the neuron receptive field, and the DNA sequence is optimized by using the genetic algorithm. The optimization goal is the neuron activation value of the DNA sequence. The mutation of the DNA sequence in the genetic algorithm is based on the neuron activation value to the DNA sequence. The gradient size of the one-hot encoding input is sampled as a probability. In addition to maintaining the cross-exchange of DNA sequences, it is also necessary to perform cyclic shifts according to the neural network pooling layer structure. Sampling the DNA sequence of the intermediate result optimized by the genetic algorithm, and does not allow repeated DNA sequences to obtain various activated DNA sequence sets. If the number is too large, it will be divided into 20 or more activation value intervals according to the maximum activation value. , perform non-repetitive random selection of DNA sequences in each interval, and discard unselected DNA sequence samples.

2) Calculate the activation values of all the neurons in each layer of the neural network that the DNA sequences in the new set of DNA sequences can affect the neuron respectively.

For the layer where the neuron is located and the shallower layers, only the activation results of some neurons in each layer will affect the neurons in this layer. Activation values for all these neurons were calculated for each DNA sequence.

3) Divide the new set.

For the DNA sequence of the new set, start from the layer where the neuron is located, and detect from the deep layer to the shallow layer. When the maximum pooling layer is encountered, according to the pooling size K, the Kmeans algorithm is used according to the pooling layer corresponding to the sequence of the new set. A shallower layer of neuron activation value features aggregates the new set of DNA sequences into K classes, each class corresponding to the divided DNA sequence subsets. The divided DNA sequence subsets are regarded as a new set of DNA sequences, starting from the cluster occurrence layer, and then from the deep layer to the shallow layer. The sequence of the set corresponds to the activation value feature of the neurons in the shallow layer of the pooling layer, and the new set of DNA sequences is aggregated into K categories, each category corresponds to the sub-set of the divided DNA sequences, and this process is repeated until the first layer, and a large number of gather.

4) Finally, calculate the mathematical representation of the gene functional element combination module corresponding to each subset.

The calculation expression of PPM is E[E(X|Y)], that is, the expected value of the one-hot encoding of the sampling sequence under the condition of activation value. X is the random variable corresponding to the one-hot encoding of the sampling sequence, Y is the random variable represented by the activation value corresponding to the sampling sequence, and the relationship between Y and X, Y=f(X), is determined by the corresponding neuron, where random The distribution of variable Y needs to be given, which is a free variable, and the random variable X depends on the random variable Y. Here, it is recommended to select the probability density function corresponding to the distribution of Y as p(y)=2y/(A*A), and A is The maximum activation value of all sequences in a collection of DNA sequences. E[E(X|Y)] represents the expected value of the one-hot encoding of the DNA sequence samples taken under the condition of a given activation value distribution. E[E(X|Y)] is calculated for each sub-set of divided DNA sequence samples, and the combination module of gene function elements represented by the neuron can be obtained. The specific calculation method is as follows:

For any subset of DNA sequences, calculate the activation value of their neuron, obtain the maximum activation value A, and divide it into N equal parts according to the size of the maximum activation value. In each interval i (i=1, 2, ...,N) in [A*(i-1)/N,A*i/N] do the following:

Find the sequence in the subset that satisfies the activation value in the interval [A*(i-1)/N, A*i/N];

Calculate the average Vi of these sequence activation values;

Calculate the average value of each position of the one-hot code of these sequences to obtain the average matrix PPMi;

After completing the calculation in each interval, the genome functional element module for calculating the subset is:

PPM=(PPM1*V1+PPM1*V2+…+PPM*VN)/(V1+V2+…+VN); PPM is the genome functional element module corresponding to the subset. Mathematically expressed PPM can draw a WebLogo diagram as shown in Figure 2.

5) For the PPM calculated for all the subsets of DNA sequences of this neuron, generalize the storage grammar.

For all the obtained gene functional element combination modules, find the same set of functional element patterns represented as {A,B,C…}, determine the length, arrangement and relative position of the functional elements in each combination module, in general, in In the combination module, their arrangement is fixed, for example, they are all ABCABC, and the number of bases spaced between two adjacent elements is inconsistent. If in all combination modules, the distance between adjacent elements is a fixed number of bases, insert the number of bases between two adjacent elements in the arrangement, such as A-6N-B, where 6N represents a distance of 6 bases in length , and for the uncertain distance, the interval can be written and divided by brackets, such as A-[6±2N]-B. Sub-modules with relatively fixed or related functions can be marked with brackets, and modules can be nested in modules, such as: [A-6N-[B-C]]-[A-6N-[B-C]], which can be expressed according to brackets Get the syntax tree of the function module.

In a specific example, matching can be performed according to the Motif in the known database to obtain the basic elements corresponding to the relevant patterns, some of which are known to exist in the database, and some are unknown and do not exist in the database, as shown in Figure 3, including CTCF , DDIT3::CEBPA, ZEB1 and an unknown transcription factor.

According to these basic elements and their relative positions in the WebLogo diagram, the following relationship can be summarized [CTCF-[6N]-DDIT3::CEBPA]-[59±1N]-[CTCF-[6N]-DDIT3::CEBPA]According to The parenthesis relationship can generate the representation of the syntax tree shown in Figure 4.

If you are not satisfied with the result, you can repeat the process of 3), 4) and 5) on the basis of the existing set until the result is satisfactory. Doing similar operations for each subset of each neuron yields a large number of genomic functional element modules. By using this method, a large number of genomic functional element modules expressed in PPM can be extracted.

It can be understood that the embodiments of the present invention propose a general algorithm NeuronMotif for decoupling convolutional neural networks, which can decouple the convolutional neural network model used to annotate whether DNA has a specific function, and discover the The identified gene regulatory elements are assembled into modules and visualized, and the algorithm can also be used to discover and visualize patterns identified by convolutional neural networks in other problem applications. In the NeuronMotif algorithm, the mathematical statistical form of the Motif corresponding to the neuron is first defined. Then, each neuron is regarded as a latent variable model, and the source and meaning of the latent variables are classified and analyzed. Based on these new analyses and understandings of neural networks and neurons, NeuronMotif is designed to realize the decoupling of the neuron hybrid model, so as to discover the corresponding Motif and Motif combination modules (expressed by PPM) of each neuron, that is, gene regulation Representation of the component combination module. The theoretical basis is established for the extraction of the combination module of gene regulatory elements in the neural network.

According to the training and visualization method of the neural network extraction and regulation DNA combination mode proposed in the embodiment of the present invention, by obtaining DNA sequences with specific functions and DNA sequences without specific functions; annotating the two DNA sequences, which will have specific functions The one-hot encoding is used to represent the DNA sequence of the DNA sequence and the DNA sequence without a specific function; a convolutional neural network is built, the one-hot encoding of the labeled DNA sequence is used as input, and the corresponding DNA sequence is labeled as the fitting value of the convolutional neural network output. , train the convolutional neural network so that the convolutional neural network can recognize DNA sequences; design and use the NeuronMotif algorithm to decouple the trained convolutional neural network, so as to discover the Motif and Motif combination modules corresponding to each neuron, The combination module of gene regulatory elements is obtained, represented and stored by the syntax tree of regulatory elements, which provides a new set of ideas and solutions for the extraction of combination modules of gene regulatory elements in neural networks.

Next, the training and visualization system for the neural network extraction and regulation DNA combination pattern proposed according to the embodiment of the present invention will be described with reference to the accompanying drawings.

5 is a schematic structural diagram of a training and visualization system for extracting and regulating DNA combination patterns by a neural network according to an embodiment of the present invention.

As shown in FIG. 5 , the training and visualization system for extracting and regulating DNA combination patterns by the neural network includes: an acquisition module 201 , a labeling module 202 , a training module 203 and a decoupling module 204 .

The obtaining module 201 is used for obtaining DNA sequences with specific functions and DNA sequences without specific functions.

The labeling module 202 is configured to label the two DNA sequences, and use one-hot encoding to represent the DNA sequences with specific functions and the DNA sequences without specific functions.

The training module 203 is used to build a convolutional neural network, taking the one-hot encoding of the marked DNA sequence as an input, and marking the corresponding DNA sequence as a fitting value output by the convolutional neural network, and training the convolutional neural network so that the A convolutional neural network recognizes DNA sequences.

The decoupling module 204 is used for decoupling the trained convolutional neural network through the NeuronMotif algorithm, to obtain a combination module of gene regulatory elements, and to use the regulatory element syntax tree for representation and storage.

Further, in one embodiment of the present invention, obtaining DNA sequences with specific functions and DNA sequences without specific functions, including:

Cut out DNA sequence fragments with specific functions and DNA sequence fragments without the specific functions from the biological genome annotated by biological experimental means; or

Through artificial synthesis of DNA sequence fragment molecules, any type of biological function verification experiment is performed to determine the fragment molecules with specific functions and the fragment molecules without specific functions.

It should be noted that, the foregoing explanations of the method embodiment are also applicable to the system of this embodiment, and details are not repeated here.

According to the training and visualization system of the neural network extraction and regulation DNA combination mode proposed in the embodiment of the present invention, by acquiring DNA sequences with specific functions and DNA sequences without specific functions; annotate the two DNA sequences, and will have specific functions The one-hot encoding is used to represent the DNA sequence of the DNA sequence and the DNA sequence that does not have a specific function; a convolutional neural network is built, and the one-hot encoding of the labeled DNA sequence is used as input, and the corresponding DNA sequence is labeled as the fitting value of the convolutional neural network output. , train the convolutional neural network so that the convolutional neural network can recognize DNA sequences; design and use the NeuronMotif algorithm to decouple the trained convolutional neural network, so as to discover the Motif and Motif combination modules corresponding to each neuron, The combination module of gene regulatory elements is obtained, represented and stored by the syntax tree of regulatory elements, which provides a new set of ideas and solutions for the extraction of combination modules of gene regulatory elements in neural networks.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (8)

1. A training and visualization method for a neural network extraction regulation and control DNA combination mode is characterized by comprising the following steps:

s1, obtaining a DNA sequence with a specific function and a DNA sequence without the specific function;

s2, labeling two DNA sequences, and representing the DNA sequence with the specific function and the DNA sequence without the specific function by using unique heat codes;

s3, building a convolutional neural network, taking the one-hot code of the labeled DNA sequence as input, labeling the corresponding DNA sequence as a fitting value output by the convolutional neural network, and training the convolutional neural network to enable the convolutional neural network to identify the DNA sequence;

s4, decoupling the trained convolutional neural network through a NeuronMotif algorithm to obtain a gene regulatory element combination module, and expressing and storing the gene regulatory element combination module by using a regulatory element syntax tree;

s41, for a neuron in the convolutional neural network, collecting a new DNA sequence set, wherein different DNA sequences in the new DNA sequence set have neuron activation values with various sizes;

s42, respectively calculating all neuron activation values of the DNA sequences in the new DNA sequence set, which can affect the neurons in each layer of the neural network;

s43, dividing the new DNA sequence set to obtain a plurality of DNA sequence subsets;

s44, calculating the mathematical expression form of the gene function component combination module corresponding to each DNA sequence subset, and expressing and storing the gene function component combination module by using a regulatory element syntax tree;

s431, for the new DNA sequence set, detecting from a deep layer to a shallow layer from the layer where the neuron is located, if a largest pooling layer is met, clustering the new DNA sequence set into K classes according to the pooling size K and the neuron activation value characteristics of the shallow layer of the pooling layer corresponding to the sequences of the new DNA sequence set by using a Kmeans algorithm, wherein each class corresponds to a divided DNA sequence subset;

s432, all the divided DNA sequence subsets are used as a DNA sequence new set, detection is carried out from a deep layer to a shallow layer from a clustering occurrence layer, if a largest pooling layer is met, the DNA sequence new set is clustered into K classes according to the pooling size K and a Kmeans algorithm according to neuron activation value characteristics of a shallow layer of the pooling layer corresponding to the sequences of the DNA sequence new set, and each class corresponds to the divided DNA sequence subsets;

and S433, repeating the step S432 to the first layer to obtain a plurality of DNA sequence subsets.

2. The method of claim 1, wherein S1 further comprises:

s11, cutting DNA sequence fragments with the specific function and DNA sequence fragments without the specific function on the biological genome marked by using a biological experimental means.

3. The method of claim 1, wherein S1 further comprises:

s12, artificially synthesizing DNA sequence fragment molecules, carrying out any type of biological function verification experiment, and determining the fragment molecules with the specific function and the fragment molecules without the specific function.

4. The method of claim 1, wherein labeling two DNA sequences comprises:

labeling the DNA sequence with the specific function as a positive sample, and labeling the DNA sequence without the specific function as a negative sample.

5. The method of claim 1, wherein S41 further comprises:

randomly generating a DNA sequence according to the size of a neuron receiving domain, optimizing the DNA sequence by using a genetic algorithm, wherein the optimization target is a neuron activation value of the DNA sequence, mutation of the DNA sequence in the genetic algorithm is sampled according to the neuron activation value and the gradient size of the one-hot coding input of the DNA sequence as probability, besides cross interchange of the DNA sequence is kept, cyclic displacement is required according to a neural network pooling layer structure, the DNA sequence of an intermediate result optimized by the genetic algorithm is sampled, the sampled DNA sequence is not repeated, and the sampled DNA sequence forms various activated DNA sequence sets.

6. The method according to claim 1, wherein the computational expression of the gene function element combination module is E [ E (X | Y) ], where X is a random variable corresponding to one-hot encoding of the sampling sequence, Y is a random variable represented by an activation value corresponding to the sampling sequence, and the relationship Y = f (X) between Y and X is determined by the corresponding neuron, where the distribution of the random variable Y needs to be given and is a free variable, and the random variable X depends on the random variable Y.

7. A training and visualization system of a neural network extraction-regulatory DNA combination pattern, characterized in that the training and visualization method for realizing the neural network extraction-regulatory DNA combination pattern of any one of claims 1 to 5 comprises:

the acquisition module is used for acquiring a DNA sequence with a specific function and a DNA sequence without the specific function;

a labeling module for labeling two DNA sequences and representing the DNA sequence with a specific function and the DNA sequence without the specific function by using unique heat codes;

the training module is used for building a convolutional neural network, taking the one-hot code of the labeled DNA sequence as input, labeling the corresponding DNA sequence as a fitting value output by the convolutional neural network, and training the convolutional neural network so that the convolutional neural network can identify the DNA sequence;

and the decoupling module is used for decoupling the trained convolutional neural network through a NeuronMotif algorithm to obtain a gene regulatory element combination module, and expressing and storing the gene regulatory element combination module by utilizing a regulatory element syntax tree.

8. The system of claim 7, wherein obtaining a DNA sequence having a specific function and a DNA sequence not having the specific function comprises:

intercepting DNA sequence fragments with the specific functions and DNA sequence fragments without the specific functions on a biological genome marked by using a biological experimental means; or

Artificially synthesizing DNA sequence fragment molecules, performing any type of biological function verification experiment, and determining the fragment molecules with the specific function and the fragment molecules without the specific function.

Images