CN112613603B - Neural network training method based on amplitude limiter and application thereof - Google Patents

Abstract

The invention discloses a limiter-based neural network training method and its application. The neural network training method includes: using experimental data as a training set to train a neural network; value; set the upper and lower limits of the limiter, change the prediction results exceeding the upper limit of the limiter to the upper limit of the limiter, change the prediction results lower than the lower limit of the limiter to the lower limit of the limiter, and change the modified The results of the training set are included in the training set; the neural network is retrained through the new training set; if the prediction results of the neural network are within the upper and lower limits of the limiter, the training ends; Within the upper and lower limits of the limiter. The invention has the advantage of establishing a neural network with higher prediction accuracy under the condition of less experimental data.

Description

Neural Network Training Method Based on Limiter and Its Application

technical field

The invention belongs to the technical field of artificial neural network prediction, relates to a neural network training method, in particular to a limiter-based neural network training method and its application.

Background technique

At present, the measurement of many physical and chemical properties is difficult and costly, and the data obtained from experiments are discrete points, which is difficult to meet the needs of the industry; the research idea of physical and chemical properties is basically to establish their theoretical calculation models through experimental data. Compared with the traditional prediction model, artificial neural network has the advantages of strong nonlinear processing ability and high prediction accuracy, and has been widely used in the prediction of physical and chemical properties. When applying a neural network, it is necessary to use a large amount of experimental data to train the neural network to learn the characteristics of different states and predict the required physical and chemical properties.

However, for many substances, there is little data on their physical and chemical properties, and the neural network trained by a small data set has the problem of low prediction accuracy. How to obtain a neural network with high prediction accuracy through limited experimental data training is a technical problem that needs to be solved urgently.

Contents of the invention

The object of the present invention is to provide a limiter-based neural network training method and its application, so as to solve one or more technical problems above. The method of the invention can improve the problem of low prediction accuracy of the neural network obtained through training of small data sets, and improve the prediction accuracy of the neural network obtained through training.

To achieve the above object, the present invention adopts the following technical solutions:

A kind of neural network training method based on limiter of the present invention comprises the following steps:

Step 1, using the pre-acquired experimental data as the training set, training the neural network to be trained, and obtaining the trained neural network; wherein, the experimental data includes experimental data for predicting required feature quantities and to be measured;

Step 2, predict the value to be measured under unknown conditions through the trained neural network, and obtain the prediction result;

Step 3, set the upper and lower limits of the limiter, and determine the rationality of the prediction results obtained in step 2 through the limiter; if the prediction results are all within the upper and lower limits of the limiter, the training ends; otherwise, change the limit If the prediction result is outside the range of the upper and lower limits of the detector, obtain a new training set and jump to step 4;

Step 4, retrain the neural network with the new training set, obtain the new trained neural network and jump to step 5;

Step 5, based on the trained neural network obtained in step 4, repeat step 2 and step 3.

A further improvement of the present invention is that, in step 3, the specific steps for obtaining a new training set include:

Change predictions that exceed the upper limit of the limiter to the upper limit of the limiter, and change predictions that are lower than the lower limit of the limiter to the lower limit of the limiter;

Incorporate the modified results into the training set as a new training set.

A further improvement of the present invention is that the output of the neural network is a normalized value.

A further improvement of the present invention is that the input of the neural network is the required feature quantity affecting the prediction of the quantity to be predicted, and the output is the quantity to be predicted.

A further improvement of the present invention is that, in step 4, when using the new training set to retrain the neural network, the influence weight of the non-experimental data on the structure and parameters of the neural network is multiplied by a coefficient less than 1.

A further improvement of the present invention is that in step 1 or step 4, during the training process of the neural network, an optimization algorithm is used to optimize the internal parameters of the neural network according to the selected loss function.

The application of the neural network training method based on the limiter of the present invention is used for the prediction of the natural gas compression factor.

A further improvement of the present invention is that the required characteristic quantities for prediction include: the content of eight components of methane, ethane, propane, butane, pentane, hexane, heptane, and octane in temperature, pressure and natural gas; The quantity is the compressibility factor of natural gas;

The neural network is a forward neural network, including an input layer, a hidden layer and an output layer; wherein, each layer has a plurality of neurons, each neuron has its own bias, weight and activation function, and each neuron in the layer The elements are independent of each other; the feature value input from the input layer, after the calculation of each hidden layer, finally reaches the output layer;

The output of the forward neural network is normalized, and the expression of normalization is:

In the formula, X is the initial value of the input, X _min and X _max are the minimum and maximum values of all inputs, and x is the normalized value of the input;

The transfer formula between each layer of the neural network is:

为具有m个神经元的第k+1层中第j个神经元对于上一层也即具有n个神经元的第k层第i个神经元a_i ^k的权重，b_j ^k+1为第k+1层中第j个神经元的偏置，

为第k+1层中第j个神经元的输出；In the formula,

is the weight of the j-th neuron in the k+1th layer with m neurons to the weight of the i-th neuron a i ^k in the upper layer, that is, the k _- th layer with n neurons, and b _j ^k+1 is The bias of the jth neuron in the k+1th layer,

is the output of the jth neuron in the k+1th layer;

The activation function used is ReLU or Tanh function, and the expression is:

或

or

为第k+1层中第j个神经元的输出，a_j ^k+1为激活函数的函数值。In the formula,

is the output of the jth neuron in the k+1th layer, and a _j ^k+1 is the function value of the activation function.

A further improvement of the present invention is that the error function is used as the optimization target, and the error function is:

In the formula, MSE is the mean square error, f(x) is the output value of the neural network, Y is the actual value; M is the number of all data.

After the error calculation, the structure and weight of the neural network are adjusted through an optimization algorithm.

The further improvement of the present invention lies in that the upper limit and the lower limit of the limiter are 1.66 and 0.36 respectively.

Compared with the prior art, the present invention has the following beneficial effects:

Compared with the existing method, the present invention has the advantage of establishing a higher prediction accuracy neural network with less experimental data; specifically, the present invention can make the neural network during training by presetting the theoretical range of the predicted value. In addition to a small amount of experimental data, other information that can support training is obtained; at the same time, the technical solution of the present invention corrects the output of the neural network according to the theoretical range, and puts the corrected value into the training set together with the experimental data, and repeats it many times until The output of the neural network all conforms to the theoretical range of the predicted value, which can give full play to the value of unlabeled data, so that when the neural network predicts data points whose input features are outside the range of the training set, the predicted value has a certain value relative to each input feature. Relatively correct trend of change, especially when the experimental data is less and the relationship between the predicted value and the input feature can be regarded as an approximate increasing convex function or decreasing concave function (similar logarithmic function); in addition, the present invention can be very Greatly improve the prediction accuracy to several times or even dozens of times that of traditional neural networks.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art; obviously, the accompanying drawings in the following description are For some embodiments of the present invention, those skilled in the art can also obtain other drawings based on these drawings without creative effort.

Fig. 1 is in the embodiment of the present invention, the structural representation of feed-forward neural network;

Fig. 2 is a schematic flow chart of a limiter-based neural network training method in an embodiment of the present invention;

Fig. 3 is a schematic diagram of the comparison of the prediction effect between the method of the present invention and the traditional feedforward neural network in the embodiment of the present invention.

Detailed ways

In order to make the purpose, technical effects and technical solutions of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention; obviously, the described embodiments It is a part of the embodiment of the present invention. Based on the disclosed embodiments of the present invention, other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall all fall within the protection scope of the present invention.

Please refer to Fig. 1 and Fig. 2, the embodiment of the present invention provides a kind of neural network training method based on limiter, application of this method can improve the problem of low prediction accuracy of the neural network obtained from the training of small data sets, and improve the accuracy of neural network prediction. precision. Described neural network training method specifically comprises the following steps:

Step 1, using the experimental data containing the feature quantities required for prediction and the measurement to be predicted as the training set to train the neural network;

Step 2, predicting the value to be measured under unknown conditions through the trained neural network;

Step 3, set the upper and lower limits of the limiter, determine the rationality of the prediction results through the limiter and change the unreasonable results; the specific method can be to change the prediction results exceeding the upper limit of the limiter to the upper limit of the limiter , change the prediction results below the lower limit of the clipper to the lower limit of the clipper, and incorporate the modified results into the training set;

Step 4, retrain the neural network through the new training set;

Step 5, repeat step 2, if the prediction results of the neural network are within the upper and lower limits of the limiter, then end the training; otherwise, return to step 3.

In yet another embodiment of the present invention, the output of the neural network is a normalized value.

In yet another embodiment of the present invention, the input of the neural network is the feature quantity affecting the quantity to be predicted, and the output is the quantity to be predicted.

In yet another embodiment of the present invention, when using a new training set to retrain the neural network, the influence weight of the non-experimental data on the structure and parameters of the neural network is multiplied by a coefficient less than 1.

In another embodiment of the present invention, during the training process of the neural network, the loss function is used as the evaluation standard of the model quality during the training process of the neural network, and various internal parameters of the neural network are optimized by an optimization algorithm. until a sufficiently high prediction accuracy is obtained.

In yet another embodiment of the present invention, the upper limit and the lower limit of the limiter can be determined according to the range of experimental data.

The application of the neural network training method based on the limiter in the embodiment of the present invention is applied to the prediction of the natural gas compression factor, which specifically includes: through 10 features, including temperature, pressure and methane, ethane, propane, butane, pentane in natural gas The content of the eight components of alkanes, hexanes, heptanes and octanes can be used to predict the compressibility factor of natural gas.

The feedforward neural network consists of an input layer, a hidden layer, and an output layer. There are multiple neurons in each layer, and each neuron has its own bias, weight, and activation function, as shown in Figure 1. The feature value is input from the input layer, and finally reaches the output layer through the calculation of each hidden layer, and each neuron in the layer is independent of each other.

The output of the feed-forward neural network is normalized so that the error will not increase with the increase of the output. The formula for normalization processing is as follows:

In the formula, X is the initial value of the input, X _min and X _max are the minimum and maximum values of all inputs, and x is the normalized value of the input

The transfer formula between each layer of the neural network is

为具有m个神经元的第k+1层中第j个神经元对于上一层也即具有n个神经元的第k层第i个神经元a_i ^k的权重，b_j ^k+1为第k+1层中第j个神经元的偏置，

为第k+1层中第j个神经元的输出；In the formula,

is the weight of the j-th neuron in the k+1th layer with m neurons to the weight of the i-th neuron a i ^k in the upper layer, that is, the k _- th layer with n neurons, and b _j ^k+1 is The bias of the jth neuron in the k+1th layer,

is the output of the jth neuron in the k+1th layer;

The activation function used is ReLU or Tanh function:

为第k+1层中第j个神经元的输出，a_j ^k+1为激活函数的函数值。In the formula,

is the output of the jth neuron in the k+1th layer, and a _j ^k+1 is the function value of the activation function.

After the neural network initializes random parameters, the forward operation is performed.

The embodiment of the present invention adopts the following error function as the optimization target

In the formula, MSE is the mean square error, f(x) is the output value of the neural network, Y is the actual value; M is the number of all data. After the error calculation, the structure and weight of the neural network are adjusted through an optimization algorithm.

The application process of the neural network training method based on the limiter is shown in Figure 2, including: for the compression factor of natural gas, the maximum value and the minimum value of the embodiment of the present invention are used as the upper limit and the lower limit of the limiter according to the existing experimental data, and the maximum The value is 1.66 and the minimum value is 0.36.

Please refer to Fig. 3, Fig. 3 has compared the prediction result of the neural network that method of the present invention and traditional neural network training method obtain, and abscissa among this figure is original data value, and ordinate is predicted by traditional neural network and the present invention For each data point, the closer to the diagonal (that is, the predicted value is equal to the experimental value), the better the accuracy. It can be seen that the technical solution of the present invention can effectively improve the prediction performance of the neural network.

In summary, the present invention discloses a method for training a neural network based on a limiter. The method comprises the following steps: using the experimental data containing the required feature quantity for prediction and the experimental data to be measured as a training set to train the neural network; Predict the value to be measured under unknown conditions; set the upper and lower limits of the limiter, determine the rationality of the prediction results through the limiter, and change the prediction results that exceed the upper limit of the limiter to the upper limit of the limiter, and change the low The prediction results based on the lower limit of the limiter are changed to the lower limit of the limiter, and the modified results are included in the training set; the neural network is retrained through the new training set; if the prediction results of the neural network are all within the upper and lower limits of the limiter, Then end the training, otherwise continue to use the limiter to change the prediction results, create a new training set, retrain the neural network, and repeat this process until the prediction results of the neural network are within the upper and lower limits of the limiter. Compared with the existing method, the present invention has the advantage of establishing a neural network with higher prediction accuracy under the condition of less experimental data.

The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art can still modify or equivalently replace the specific embodiments of the present invention. , any modifications or equivalent replacements that do not deviate from the spirit and scope of the present invention are within the protection scope of the claims of the present invention pending application.

Claims (5)

1. a method for predicting natural gas compressibility factor, is characterized in that, comprises the following steps: The characteristic quantities required for prediction include: temperature, pressure, and the content of eight components in natural gas: methane, ethane, propane, butane, pentane, hexane, heptane, and octane; the compression factor of natural gas to be measured; The neural network used for prediction is a feed-forward neural network, including an input layer, a hidden layer, and an output layer; each layer has multiple neurons, each neuron has its own bias, weight, and activation function. The neurons are independent of each other; the eigenvalues input from the input layer are calculated by each hidden layer and finally reach the output layer; The output of the forward neural network is normalized, and the expression of normalization is:

In the formula, X is the initial value of the input, X _min and X _max are the minimum and maximum values of all inputs, respectively, and x is the normalized value of the output; The transfer formula between each layer of the neural network is:

In the formula,

is the weight of the jth neuron in the k+1th layer with m neurons to the i kth neuron a i ^k in the kth layer with n neurons, and b _j k ₊ ¹ is the k+1th layer The bias of the jth neuron in ,

is the output of the jth neuron in the k+1th layer; The activation function used is ReLU or Tanh function, and the expression is:

or

In the formula,

is the output of the jth neuron in the k+1th layer, a _j ^k+1 is the function value of the activation function; The neural network training method includes the following steps: Step 1, using the pre-acquired experimental data as the training set, training the neural network to be trained, and obtaining the trained neural network; wherein, the experimental data includes experimental data for predicting required feature quantities and to be measured; Step 2, predict the value to be measured under unknown conditions through the trained neural network, and obtain the prediction result; Step 3, set the upper and lower limits of the limiter, and determine the rationality of the prediction results obtained in step 2 through the limiter; if the prediction results are all within the upper and lower limits of the limiter, the training ends; otherwise, change the limit If the prediction result is outside the range of the upper and lower limits of the detector, obtain a new training set and jump to step 4; Step 4, retrain the neural network with the new training set, obtain the new trained neural network and jump to step 5; Step 5, based on the trained neural network obtained in step 4, repeat step 2 and step 3. 2. the prediction method of a kind of natural gas compressibility factor according to claim 1, it is characterized in that, in step 3, described otherwise changes the prediction result outside limiter upper and lower limit scope, the specific step of obtaining new training set comprises : Change predictions that exceed the upper limit of the limiter to the upper limit of the limiter, and change predictions that are lower than the lower limit of the limiter to the lower limit of the limiter; Incorporate the modified results into the training set as a new training set. 3. the prediction method of a kind of natural gas compressibility factor according to claim 1, it is characterized in that, in step 4, when utilizing new training set to retrain neural network, non-experimental data is to described neural network structure and parameter The influence weight is multiplied by a factor less than 1. 4. the prediction method of a kind of natural gas compressibility factor according to claim 1, is characterized in that, adopts loss function as optimization target, and loss function is:

In the formula, MSE is the mean square error, f(x) is the output value of the neural network, Y is the actual value; M is the number of all data; After the error calculation, the structure and weight of the neural network are adjusted through an optimization algorithm. 5. The method for predicting a natural gas compression factor according to claim 1, wherein the upper and lower limits of the limiter are 1.66 and 0.36 respectively.

Images